An anti-forensic scheme on computer graphic images and natural images using generative adversarial networks

1.   Introduction

Data security and privacy has become the crucial concern ^[1]. At the same time, image tampering technology and image processing function are becoming more and more powerful, which gradually reduces the authenticity and reliability of digital images. In order to detect tampered or forged digital images, forensics algorithms ^[2,3,4,5,6] have been constantly improved. As one of significant research of forensics, forensic algorithms on natural images (NI) and computer graphic images (CGI) supply the forensics for the source of the suspicious image. NIs are photos taken by the terminal devices, and CGIs are the productions of computer generation. With the development of algorithms related to computer graphics, CGI can be devised to an extent that is indistinguishable by the human eye. Due to the increasing computing power of smart devices, high-quality CGI is also easy to make, and CGI will be forged as NI to obtain the illegal benefit. Therefore, the forensic on CGI and NI will be significant in protecting data security and personal property. As a matter of fact, effective research results have been obtained in this field. In addition, Photo Response Non-Uniformity (PRNU) is an important feature for image forensics tasks. Peng et al. ^[7] employed photo response non-uniformity (PRNU) as a breakthrough research point to design effective algorithms. The best error rate of the detection in their presented experiment is 5.71%. Long et al. ^[8] improved the forensics algorithm in ^[7] further by using binary similarity of PRNU as the measures and achieved a higher detection rate of 99.83%. There are achievements of forensics on NI and CGI with the development of deep leaning. Yang et al. ^[2] proposed a contrast enhancement forensics algorithm by P-CNN and H-CNN those are two convolutional neural networks. Especially, the deeper convolutional neural networks (DCNN) ^[9] based approaches make the forensics become Intelligent. DCNN provide an environment that pulling the feature extraction and training together. In this scheme, it is only required to adjust the architecture of DCNN to make it suitable for a forensic task. Modifying ResNet ^[10] by adding a pre-process layer for enhancing effective features for forensic on NI and CG, Cui et al. ^[11] achieved the classifying with the average accuracy of 98%. Quan et al. ^[12] proposed a CNN-based architecture consists of four-layer convolutional neural networks and two-layer fully-connected networks for forensic on NI and CGI with the classification accuracy of 98.50%. In common, the dataset using in the above approaches is Columbia Photographic Images dataset ^[13], which is conducted aiming at the study of NI and CGI classification. it does not contain the latest CGI samples, especially those generated from generative adversarial networks (GAN). Besides, Wang et al. ^[14] provide an effective approach to identifying computer generated images in color quaternion wavelet domain. In this study, we combined the fixed filters in the pre-processing layer of ^[11] and the generic convolutional neural networks in ^[12] as the discriminate network in our proposed GAN-based network.

On the contrary, the research on anti-forensics is also constantly improving. These research will find the defect of the existing forensics approach. The contrary to the forensics method is achieved by proposing a corresponding resistance algorithm for a certain detection method. The significance of anti-forensic algorithm is that it can effectively prevent the intruder from judging under forensic methods. At the same time, the study of anti-forensics helps to help forensic research by upright researchers. There are several kinds of main anti-forensics methods ^{[15,16,17,18,19]}. Anti-forensic of JPEG image composition ^[20] is a practical and effective approach for common operation ^[21,22]. This approach makes the compressed images undetectable by estimating the distribution of coefficients. Li et al. ^[23] propose a multiclass classification method to classify the common image operations. In addition, they propose a compact universal feature set. There are also schemes to actively attack the detection method ^[24]. In order to guaranty information security and protecting copyright, the field of information hiding and forensics has been studied. In the field of information hiding, there have been many forensics research results of steganalysis algorithms countering steganography ^[25,26]. In recent years, the attributes of deep learning have also been applied to information hiding. Particularly, the adversarial training strategy in GAN provides a pattern that could be deployed in the field of anti-forensics. Actually, many algorithms utilized the similarity between GAN ^[27] and information hiding to combine steganography with GAN ^[28,29,30] which employ steganalysis networks as one of the discriminators. In addition, Meng et al. ^[31] proposed to use object detection method such as Faster R-CNN ^[32] to select the safe hiding area, so as to make steganography more secure and robust.

In this paper, we enhanced the capability of anti-detection on NI and CGI forensics by using the GAN-based model with the adversarial concept. The contributions of this paper include: a. We define a GAN-based architecture with reforming the discriminator for NI and CGI anti-forensics. b. We provide adversarial training to generate photorealistic computer images which can fool the detector to some extent.

2.   Related works

2.1.   Generative adversarial networks

With the wide application of deep learning, suitable data sets are needed by various networks. In order to solve the problem of insufficient data sets, GAN was proposed by Goodfellow et al. GAN consists of two sub-networks: generator and discriminator. as shown in Figure 1. Firstly, random noise is fed into the generator. Then the generator generates a false image with a randomly initialed image distribution and put the false image into the discriminator. At the same time, the real image is also fed into the discriminator as the other group of input. Through the confrontation between generator and discriminator, a relatively natural image is generated. The objective function of the whole network is denoted as:

Figure 1.  The specific structure of the proposed architecture. The full name of the abbreviations NI and ICGI is natural images and improved computer graphic images. Each block represents a group of feature maps.

DownLoad: Full-Size Img PowerPoint

where $x$ is the randomly sampled real data, and $z$ is the initial noise generated randomly. The whole process can be regarded as two optimization problems, one is the process of optimizing the sub-network $D$ that is denoted in equation (2.2), the other is the process of optimizing the sub-network $G$ that is denoted in equation (2.3):

The training goal of the first step is to update the sub-network by ascending the stochastic gradient of it:

As the loss value of the whole network decreases gradually in the training process, the training model will approach the global optimum to a certain extent. Since the distribution of the ultimate optimization is in the format of digital image on the research we pay attention to, the form of the generated data is image.

Further, in order to solve the problem of mode collapse in ^[27]. Gulrajani et al. ^[33] transformed the indicators of measuring similarity from K-L divergence to J-S divergence shown in the equation (2.5) and equation (2.6).

The value of the equation (2.5) and equation (2.6) indicates the distance between the distributions of ${\mathbb{P}_r}$ and ${\mathbb{P}_g}$. The term ${\mathbb{P}_m}$ of equation (2.6) is $\left( {{\mathbb{P}_r}{\text{ + }}{\mathbb{P}_g}} \right){\text{/2}}$. Then the improved algorithm named WGAN-GP introduced the gradient penalty as:

2.2.   Image forensics based on deep learning

In order to guaranty information security and protecting copyright, the field of information hiding and forensics has been studied. In the field of information hiding, there have been many forensics research results of steganalysis algorithms countering steganography ^[25,26]. In essence, deep learning is to construct a structure model with multiple hidden layers. By training large-scale sample data, representative feature information can be obtained, and new samples can be classified and regressed. The goal of deep learning is to enable machines to have the ability of analysis and learning, and to recognize data information such as words, images and sounds. In recent years, the attributes of deep learning have also been applied to information hiding. The CNN-based network applied to steganalysis proposed by Xu el at. ^[34] benefited from the BN layer. The normalization process of the input feature maps is denoted as:

where $k$ indicates the identifier of feature maps, $\mu$ indicates the average value and $\sigma$ indicates the variance, then $x_{n, j, k}^{k'}$ denotes the normalized feature maps.

With the power of the feature exaction of CNN, Baroffio et al. ^[35] presented the CNN-based approach for camera model identification. The Convolution process of the CNN is shown as:

where $\otimes$ represent the convolution operation, $x$ denotes the feature maps, $w$ and $b$ denote the kernel and bias in the networks respectively, $i$ and $j$ represent the identifier of the output feature maps, $l$ is the identifier of the output layer.

3.   The proposed methodology

3.1.   Network structures

In this part, we present the proposed method named as generative adversarial networks for anti-forensics. The effective adversarial concept is the main architecture to the approach.

The whole scheme is composed of two main subnets. The detailed structures are shown in Figure 1. The specific structure of the proposed networks shown in Table 1. The Conv layers consist of the sub-network as the discriminator, and the Deconv layers consist of the sub-network as the generator. The residual block in ^[11] has shown the effective results on the task of distinguishing NI and CGI. We add the convolutional residual block as each convolution layer inspired by ^[11]. Each of the residual blocks consists of two-layer convolution layers and an activation function between them. We choose the leaky rectified linear unit (Leaky Relu) as the activation function. The structure of the residual block is shown in Figure 2. The weight normalization (WN) ^[36] is followed after each convolutional layer to facilitate to stabilize the training process and make the generated images more realistic.

Table 1.  The configuration of the proposed GAN in detail. Conv layers represent convolutional layers. DeConv layers is transposed Conv layers.

Layers	Input layer size	Stride	Padding	Kernel size
Conv1	3	2	1	4
Conv2	64	2	1	4
Conv3	128	2	1	4
Conv4	256	2	1	4
Conv5	384	2	1	4
Conv6	512	1	0	5
DeConv1	512	1	0	5
DeConv2	384	2	1	4
DeConv3	256	2	1	4
DeConv4	128	2	1	4
DeConv5	64	2	1	4
DeConv6	3	2	1	4

---

 | Show Table

DownLoad: CSV

Figure 2.  The structure in detail of the residual block.

DownLoad: Full-Size Img PowerPoint

As the main part of the proposed approach, we define a sub-net of discriminator as the forensics to force the generated images to fool the detector. There are a couple of Sobel Filters ${F_1}$ and ${F_2}$ referenced in ^[11] as the kernels in the first convolutional layers of the discriminative network. ${F_1}$ and ${F_2}$ concentrate more on the texture and edge information of the input images. The two filters are 3×3 arrays as shown in equation (3.1). The generated images by the model of the generator are collected as an amplified image set ${I_G}$ of CGI. Then the discriminator is fed with the images in ${I_G}$ as fake images and the NI images as real image. The ultimately generated images are expected with the characteristic of resisting detector owing to adopt the forensic component.

3.2.   Loss design

In this segment, we demonstrate the loss functions for the two sub-networks. The target of the generator is evaluating the images with high-level qualities and the characteristics of NI to fool the forensics model on NI and CG. With the aid of the generation of diversity and stabilization by adding weight normalization in the proposed generative adversarial networks, we reference WGAN-GP ^[33] to design the loss function. Here the objective functions of the two sub-networks are shown in equation (3.2) and equation (3.3).

$\widehat x$ is the random interpolation sampling, and the specific expression is:

The function ${f_w}$ means Wasserstein distance which is shown as:

4.   Experiments

4.1.   Experimental setting

The deep learning framework implemented in the experiment is PyTorch which is integrated with Python. The hardware environment is composed of Intel i9 CPU with 32GB memory and NVIDIA Geforce 1080Ti GPU with 12GB memory. The NI data set is CelebA ^[37] face data set as the generating target during training the model. The open access data set is usually used for generating images in the research of GAN, which contains 202,599 images extracted from 10,177 identities. The aligned version of the data set clips the image to the size of 178 × 218 aligning with the main part of the face.

4.2.   Training setting

In the training process, we set the learning rate of $2 \times {10^{ - 5}}$. The size of each batch is set to 8, and the resolution of the generated image is set to ${\text{160}} \times {\text{160}}$. The loss function for the training process is sigmoid binary cross entropy loss:

where $x$ is the output of the model which represents the generated images and y is the target sample image. We use the optimization function of root mean square prop (RMSprop) ^[38] for descending the gradient. The parameters updated formula is:

Where, ${S_{dW}}$ and ${S_{db}}$ denote the gradient momentum for weights and bias respectively. The formulas of ${S_{dW}}$ and ${S_{db}}$ are as follows:

4.3.   valuation

The model of the network is trained after 400,000 iterations. The generated samples by model after the well-trained model are shown in Figure 3. The loss of the whole process of the training evolution is shown in Figure 4. The loss value consists of two parts in our experiments, one is the loss of the real data and the other is the loss on the fake data. The two parts are shown in Figure 5 and Figure 6. The anti-forensics on NI and CGI is verified since the curve of the loss is gradually declined, which means the discriminator cannot distinguish the inputted images correctly with the adversarial training between the generator and the discriminator.

Figure 3.  The random sampled generated images by the well-trained model of the proposed approach.

DownLoad: Full-Size Img PowerPoint

Figure 4.  The loss curve of the test evolution during the iterations of 400,000.

DownLoad: Full-Size Img PowerPoint

Figure 5.  The loss curve of the discriminator on the real data during the training evolution.

DownLoad: Full-Size Img PowerPoint

Figure 6.  The loss curve of the discriminator on the fake data during the training evolution.

DownLoad: Full-Size Img PowerPoint

To further verify the effectiveness of the trained model, we randomly select 200 test samples as generated targets for testing the visual effect and the characteristics of anti-forensics during the training process of each step of the generation and recording the testing loss value on the model during the training process. The samples target and the generated images of the test process are shown in Figure 7. The loss value during the test process is shown in Figure 8. With the increase of iteration steps, the loss value decreases gradually. The feature of anti-forensics to invalidate the CNN-based forensics algorithms ^[11,12] in the generated images is verified for our proposed approach.

Figure 7.  The randomly selected target real natural images (left), and the generated images by the proposed approach targeting the randomly selected real natural images (right).

DownLoad: Full-Size Img PowerPoint

Figure 8.  The loss curve graph of the discriminator during the test evolution.

DownLoad: Full-Size Img PowerPoint

5.   Conclusion

In this paper, we present an anti-forensics scheme for high-performance CGI generation based on GAN. We first analyze deep learning in the field of forensics including the latest CNN-based forensics algorithms and the generative adversarial learning model. By comparing the similarities between forensics and GAN, we found that the adversarial concept is suitable for anti-forensics tasks on distinguishing NI and CGI. At the same time, the anti-forensics scheme we proposed is available at resisting the uncertificated forensics. The generated images by our improving will treat the detector effectively with the trained model.

Acknowledgments

This work is supported by the National Key R & D Program of China under grant 2018YFB1003205; by the National Natural Science Foundation of China under grant U1836208, U1536206, U1836110, 61602253, 61672294; by the Jiangsu Basic Research Programs-Natural Science Foundation under grant numbers BK20181407; by the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD) fund; by the Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET) fund, China.

Conflict of interest

The authors declare that there are no actual or potential conflicts of interest in relation to this article.