Export file:


  • RIS(for EndNote,Reference Manager,ProCite)
  • BibTex
  • Text


  • Citation Only
  • Citation and Abstract

A convolutional neural network-based linguistic steganalysis for synonym substitution steganography

1 Hunan Provincial Key Laboratory of Intelligent Processing of Big Data on Transportation, Changsha University of Science and Technology, Changsha 410114, Hunan, China
2 School of Computer and Communication Engineering, Changsha University of Science and Technology, Changsha 410114, Hunan, China
3 Hunan Provincial Key Laboratory of Smart Roadway and Cooperative Vehicle-Infrastructure Systems, Changsha University of Science and Technology, Changsha 410114, Hunan, China
4 Department of Computer Science, University of Central Arkansas, Conway, AR, USA 72035
5 Hunan Branch of CNCERT/CC, Changsha 410004, Hunan, China

Special Issues: Information Multimedia Hiding & Forensics based on Intelligent Devices

In this paper, a linguistic steganalysis method based on two-level cascaded convolutional neural networks (CNNs) is proposed to improve the system’s ability to detect stego texts, which are generated via synonym substitutions. The first-level network, sentence-level CNN, consists of one convolutional layer with multiple convolutional kernels in different window sizes, one pooling layer to deal with variable sentence lengths, and one fully connected layer with dropout as well as a softmax output, such that two final steganographic features are obtained for each sentence. The unmodified and modified sentences, along with their words, are represented in the form of pre-trained dense word embeddings, which serve as the input of the network. Sentence-level CNN provides the representation of a sentence, and can thus be utilized to predict whether a sentence is unmodified or has been modified by synonym substitutions. In the second level, a text-level CNN exploits the predicted representations of sentences obtained from the sentence-level CNN to determine whether the detected text is a stego text or cover text. Experimental results indicate that the proposed sentence-level CNN can effectively extract sentence features for sentence-level steganalysis tasks and reaches an average accuracy of 82.245%. Moreover, the proposed steganalysis method achieves greatly improved detection performance when distinguishing stego texts from cover texts.
  Article Metrics

Keywords steganalysis; steganography; synonym substitution; word embedding; convolutional neural network

Citation: Lingyun Xiang, Guoqing Guo, Jingming Yu, Victor S. Sheng, Peng Yang. A convolutional neural network-based linguistic steganalysis for synonym substitution steganography. Mathematical Biosciences and Engineering, 2020, 17(2): 1041-1058. doi: 10.3934/mbe.2020055


  • 1 Y. L. Liu, H. Peng and J. Wang, Verifiable diversity ranking search over encrypted outsourced data, CMC-Comput. Mater. Con., 55 (2018), 37-57.
  • 2 L. Y. Xiang, Y. Li, W. Hao, et al., Reversible natural language watermarking using synonym substitution and arithmetic coding, CMC-Comput. Mater. Con., 55 (2018), 541-559.
  • 3 H. M. Meral, B. Sankur, A. S. Ozsoy, et al., Natural language watermarking via morphosyntactic alterations, Comput. Speech Lang., 23 (2009), 107-125.
  • 4 C. M. Taskiran, M. Topkara and E. J. Delp, Attacks on lexical natural language steganography systems, Proceed. SPIE, 6072 (2006), 607209-607209-9.
  • 5 Z. L. Chen, L. S. Huang, H. B. Miao, et al., Steganalysis against substitution-based linguistic steganography based on context clusters, Comput. Electr. Eng., 37 (2011), 1071-1081.
  • 6 Z. L. Chen, L. S. Huang and W. Yang, Detection of substitution-based linguistic steganography by relative frequency analysis, Digit. Invest., 8 (2011), 68-77.
  • 7 L. Y. Xiang, X. M. Sun, G. Luo, et al., Linguistic steganalysis using the features derived from synonym frequency, Multimed. Tools Appl., 71 (2014), 1893-1911.
  • 8 L. Y. Xiang, J. M. Yu, C. F. Yang, et al., A word-embedding-based steganalysis method for linguistic steganography via synonym-substitution, IEEE Access, 6 (2018), 64131-64141.
  • 9 Z. S. Yu, L. S. Huang, Z. L. Chen, et al., Steganalysis of synonym-substitution based natural language watermarking, Int. J. Mult. Ubiquit. Eng., 4 (2012), 21-34.
  • 10 Z. S. Yu, L. S. Huang, Z. L. Chen, et al., Detection of synonym-substitution modified articles using context information, Second International Conference on Future Generation Communication and Networking, (2008), 134-139.
  • 11 Y. T. Chen, J. Xiong, W. H. Xu, et al., A novel online incremental and decremental learning algorithm based on variable support vector machine, Cluster Comput., Available from: https://doi.org/10.1007/s10586-018-1772-4.
  • 12 L. Y. Xiang, G. H. Zhao, Q. Li, et al., TUMK-ELM: A fast unsupervised heterogeneous data learning approach, IEEE Access, 6 (2018), 35305-35315.
  • 13 I. A. Bolshakov, A method of linguistic steganography based on collocationally-verified synonymy, International Workshop on Information Hiding, (2004), 180-191.
  • 14 C. Y. Chang and S. Clark, Practical linguistic steganography using contextual synonym substitution and a novel vertex coding method, Comput. Linguist., 40 (2014), 403-448.
  • 15 X. Yang, F. Li and L. Y. Xiang, Synonym substitution-based steganographic algorithm with matrix coding, J. Chinese Comput. Syst., 36 (2015), 1296-1300.
  • 16 H. H. Hu, X. Zuo, W. M. Zhang, et al., Adaptive text steganography by exploring statistical and linguistical distortion, IEEE Second International Conference on Data Science in Cyberspace, (2017), 145-150.
  • 17 O. Russakovsky, J. Deng, H. Su, et al., Imagenet large scale visual recognition challenge, Int. J. Comput. Vision, 115 (2015), 211-252.    
  • 18 L. Y. Xiang, X. B. Shen, J. H. Qin, et al., Discrete multi-graph hashing for large-scale visual search, Neural Process. Lett., 49 (2019), 1055-1069.
  • 19 J. Wang, J. H. Qin, X. Y. Xiang, et al., CAPTCHA recognition based on deep convolutional neural network, Math. Biosci. Eng., 16 (2019), 5851-5861.
  • 20 N. Kalchbrenner, E. Grefenstette and P. Blunsom, A convolutional neural network for modelling sentences, preprint, arXiv:1404.2188.
  • 21 D. J. Zeng, Y. Dai, F. Li, et al., Aspect based sentiment analysis by a linguistically regularized CNN with gated mechanism, J. Intell. Fuzzy Syst., 36 (2019), 3971-3980.    
  • 22 R. H. Meng, S. G. Rice, J. Wang, et al., A fusion steganographic algorithm based on faster R-CNN, CMC-Comput. Mater. Con., 55 (2018), 001-016.
  • 23 S. Q. Tan and B. Li, Stacked convolutional auto-encoders for steganalysis of digital images, Signal and Information Processing Association Summit and Conference, (2014), 1-4.
  • 24 J. Q. Ni, J. Ye and Y. Yang, Deep learning hierarchical representations for image steganalysis, IEEE T. Inf. Foren. Sec., 12 (2017), 2545-2557.
  • 25 Y. L. Qian, J. Dong, W. Wang, et al., Deep learning for steganalysis via convolutional neural networks, Proceed. SPIE, 9409 (2015), 94090J-94090J-10.
  • 26 G. S. Xu, H. Z. Wu and Y. Q. Shi, Structural design of convolutional neural networks for steganalysis, IEEE Signal Proc. Let., 23 (2016), 708-712.
  • 27 J. S. Zeng, S. Q. Tan, B. Li, et al., Large-scale jpeg image steganalysis using hybrid deep learning framework, IEEE T. Inf. Foren. Sec., 13 (2018), 1200-1214.
  • 28 A. Krizhevsky, I. Sutskever and G. E. Hinton, Imagenet classification with deep convolutional neural networks, Proceedings of the 25th International Conference on Neural Information Processing Systems, 1 (2012), 1097-1105.
  • 29 Y. Kim, Convolutional neural networks for sentence classification, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, (2014), 1746-1751.
  • 30 J. Turian, L. Ratinov and Y. Bengio, Word representations: A simple and general method for semisupervised learning, Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, (2010), 384-394.
  • 31 G. E. Hinton, Learning distributed representations of concepts, Proceedings of the Eighth Annual Conference of the Cognitive Science Society, 1 (1986), 12-23.
  • 32 Y. Bengio, H. Schwenk, J. Sencal, et al., Neural probabilistic language models, J. Mach. Learn. Res., 3 (2003), 1137-1155.
  • 33 F. Morin and Y. Bengio, Hierarchical probabilistic neural network language model, Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics, (2005), 246-252.
  • 34 R. Collobert and J. Weston, A unified architecture for natural language processing: Deep neural networks with multitask learning, Proceedings of the 25th International Conference on Machine Learning, (2008), 160-167.
  • 35 T. Mikolov, I. Sutskever, K. Chen, et al., Distributed representations of words and phrases and their compositionality, International Conference on Neural Information Processing Systems, (2013), 3111-3119.
  • 36 B. Shen, C. W. Forstall, A. Rocha, et al., Practical text phylogeny for real-world settings, IEEE Access, 6 (2018), 41002-41012.
  • 37 D. J. Zeng, Y. Dai, F. Li, et al., Adversarial learning for distant supervised relation extraction, CMC-Comput. Mater. Con., 55 (2018), 121-136.
  • 38 R. Collobert, J. Weston, L. Bottou, et al., Natural language processing (almost) from scratch, J. Mach. Learn. Res., 12 (2011), 2493-2537.
  • 39 Y. L. Boureau, N. L. Roux, F. Bach, et al., Ask the locals: Multi-way local pooling for image recognition, 2011 International Conference on Computer Vision, (2011), 2651-2658.
  • 40 C. F. Yang, F. L. Liu, S. K. Ge, et al., Locating secret messages based on quantitative steganalysis, Math. Biosci. Eng., 16 (2019), 4908-4922.
  • 41 C. F. Yang, X. Y. Luo, J. C. Lu, et al., Extracting hidden messages of MLSB steganography based on optimal stego subset, Sci. China Inform. Sci., 61 (2018), 119103:1-119103:3.


This article has been cited by

  • 1. Fei Yu, Zinan Zhang, Li Liu, Hui Shen, Yuanyuan Huang, Changqiong Shi, Shuo Cai, Yun Song, Sichun Du, Quan Xu, Secure Communication Scheme Based on a New 5D Multistable Four-Wing Memristive Hyperchaotic System with Disturbance Inputs, Complexity, 2020, 2020, 1, 10.1155/2020/5859273
  • 2. Wentao Zhao, Xiao Liu, Qiang Liu, Jiuren Chen, Pan Li, Model Capacity Vulnerability in Hyper-Parameters Estimation, IEEE Access, 2020, 8, 21602, 10.1109/ACCESS.2020.2969276
  • 3. Haowei Li, Chunxi Zhang, Huipeng Li, Ningfang Song, White-Light Interference Microscopy Image Super-Resolution Using Generative Adversarial Networks, IEEE Access, 2020, 8, 27724, 10.1109/ACCESS.2020.2971841
  • 4. Feng Li, Xiaoyu Li, Fei Wang, Dengyong Zhang, Yi Xia, Fan He, A Novel P300 Classification Algorithm Based on a Principal Component Analysis-Convolutional Neural Network, Applied Sciences, 2020, 10, 4, 1546, 10.3390/app10041546
  • 5. Feng Li, Yi Xia, Fei Wang, Dengyong Zhang, Xiaoyu Li, Fan He, Transfer Learning Algorithm of P300-EEG Signal Based on XDAWN Spatial Filter and Riemannian Geometry Classifier, Applied Sciences, 2020, 10, 5, 1804, 10.3390/app10051804
  • 6. Feng Li, Fan He, Fei Wang, Dengyong Zhang, Yi Xia, Xiaoyu Li, A Novel Simplified Convolutional Neural Network Classification Algorithm of Motor Imagery EEG Signals Based on Deep Learning, Applied Sciences, 2020, 10, 5, 1605, 10.3390/app10051605
  • 7. Fei Yu, Hui Shen, Li Liu, Zinan Zhang, Yuanyuan Huang, Binyong He, Shuo Cai, Yun Song, Bo Yin, Sichun Du, Quan Xu, CCII and FPGA Realization: A Multistable Modified Fourth-Order Autonomous Chua’s Chaotic System with Coexisting Multiple Attractors, Complexity, 2020, 2020, 1, 10.1155/2020/5212601
  • 8. Fei Yu, Li Liu, Shuai Qian, Lixiang Li, Yuanyuan Huang, Changqiong Shi, Shuo Cai, Xianming Wu, Sichun Du, Qiuzhen Wan, Chaos-Based Application of a Novel Multistable 5D Memristive Hyperchaotic System with Coexisting Multiple Attractors, Complexity, 2020, 2020, 1, 10.1155/2020/8034196
  • 9. Fan Zhang, Fenlin Liu, Xiangyang Luo, Geolocation of covert communication entity on the Internet for post-steganalysis, EURASIP Journal on Image and Video Processing, 2020, 2020, 1, 10.1186/s13640-020-00504-8
  • 10. Ruoyu Zhang, Wenpeng Lu, Shoujin Wang, Xueping Peng, Rui Yu, Yuan Gao, Chinese clinical named entity recognition based on stacked neural network, Concurrency and Computation: Practice and Experience, 2020, 10.1002/cpe.5775
  • 11. Fei Yu, Li Liu, Hui Shen, Zinan Zhang, Yuanyuan Huang, Changqiong Shi, Shuo Cai, Xianming Wu, Sichun Du, Qiuzhen Wan, Dynamic Analysis, Circuit Design, and Synchronization of a Novel 6D Memristive Four-Wing Hyperchaotic System with Multiple Coexisting Attractors, Complexity, 2020, 2020, 1, 10.1155/2020/5904607
  • 12. Aqsa Shakeel, Toshihisa Tanaka, Keiichi Kitajo, Time-Series Prediction of the Oscillatory Phase of EEG Signals Using the Least Mean Square Algorithm-Based AR Model, Applied Sciences, 2020, 10, 10, 3616, 10.3390/app10103616
  • 13. Zhonghua Liu, Lingyun Xiang, Kaiming Shi, Kaibing Zhang, Qingtao Wu, Robust Manifold Embedding for Face Recognition, IEEE Access, 2020, 8, 101224, 10.1109/ACCESS.2020.2997953
  • 14. Nan Pan, Jiaohua Qin, Yun Tan, Xuyu Xiang, Guimin Hou, A video coverless information hiding algorithm based on semantic segmentation, EURASIP Journal on Image and Video Processing, 2020, 2020, 1, 10.1186/s13640-020-00512-8
  • 15. H. Li, S. Jin, Text Steganalysis Based on Capsule Network with Dynamic Routing, IETE Technical Review, 2020, 1, 10.1080/02564602.2020.1780959
  • 16. Wenjun Li, Yang Ding, Yongjie Yang, R. Simon Sherratt, Jong Hyuk Park, Jin Wang, Parameterized algorithms of fundamental NP-hard problems: a survey, Human-centric Computing and Information Sciences, 2020, 10, 1, 10.1186/s13673-020-00226-w
  • 17. Zhongxiang Huang, Xiangjun Jiang, Wei Hao, A Proportional-Switch Adjustment Model towards Mixed Equilibrium with Multiroute Choice Behaviour Criterion, Journal of Advanced Transportation, 2020, 2020, 1, 10.1155/2020/1269415
  • 18. Jiaohua Qin, Jing Wang, Yun Tan, Huajun Huang, Xuyu Xiang, Zhibin He, Coverless Image Steganography Based on Generative Adversarial Network, Mathematics, 2020, 8, 9, 1394, 10.3390/math8091394
  • 19. Fei Yu, Shuai Qian, Xi Chen, Yuanyuan Huang, Li Liu, Changqiong Shi, Shuo Cai, Yun Song, Chunhua Wang, A New 4D Four-Wing Memristive Hyperchaotic System: Dynamical Analysis, Electronic Circuit Design, Shape Synchronization and Secure Communication, International Journal of Bifurcation and Chaos, 2020, 30, 10, 2050147, 10.1142/S0218127420501473
  • 20. Qiang Liu, Xuyu Xiang, Jiaohua Qin, Yun Tan, Yao Qiu, Coverless image steganography based on DenseNet feature mapping, EURASIP Journal on Image and Video Processing, 2020, 2020, 1, 10.1186/s13640-020-00521-7
  • 21. Lingyun Xiang, Shuanghui Yang, Yuhang Liu, Qian Li, Chengzhang Zhu, Novel Linguistic Steganography Based on Character-Level Text Generation, Mathematics, 2020, 8, 9, 1558, 10.3390/math8091558
  • 22. Jiufen Liu, Chunfang Yang, Junchao Wang, Yanan Shi, Stego key recovery method for F5 steganography with matrix encoding, EURASIP Journal on Image and Video Processing, 2020, 2020, 1, 10.1186/s13640-020-00526-2

Reader Comments

your name: *   your email: *  

© 2020 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution Licese (http://creativecommons.org/licenses/by/4.0)

Download full text in PDF

Export Citation

Copyright © AIMS Press All Rights Reserved