First steps in the investigation of automated text annotation with pictures

J. Kent Poots; Nick Cercone

doi:10.3934/bdia.2017001

Big Data and Information Analytics, 2017, 2(2): 97-106. doi: 10.3934/bdia.2017001. Article

Export file:

Format

RIS(for EndNote,Reference Manager,ProCite)
BibTex
Text

Content

Citation Only
Citation and Abstract

First steps in the investigation of automated text annotation with pictures

J. Kent Poots, Nick Cercone

York University Dept. of Electrical Engineering and Computer Science 4700 Keele Street, Toronto, Ontario, M3J 1P3, Canada

Abstract Full Text(HTML) Figure/Table Related pages

We describe the investigation of automatic annotation of text with pictures, where knowledge extraction uses dependency parsing. Annotation of text with pictures, a form of knowledge visualization, can assist understanding. The problem statement is, given a corpus of images and a short passage of text, extract knowledge (or concepts), and then display that knowledge in pictures along with the text to help with understanding. A proposed solution framework includes a component to extract document concepts, a component to match document concepts with picture metadata, and a component to produce an amalgamated output of text and pictures. A proof-of-concept application based on the proposed framework provides encouraging results

Figure/Table

Supplementary

Article Metrics

Keywords: Natural language processing; natural language understanding; information extraction; information visualization; artificial intelligence

Citation: J. Kent Poots, Nick Cercone. First steps in the investigation of automated text annotation with pictures. Big Data and Information Analytics, 2017, 2(2): 97-106. doi: 10.3934/bdia.2017001

References:

[1] B. Coyne and R. Sproat, WordsEye:An automatic text-to-scene conversion system, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, 3(2003), 487-496.
[2] D. Genzel, K. Macherey and J. Uszkoreit, Creating a high-quality machine translation system for a low-resource language:Yiddish, (2009), Available from:www.mt-archive.info/MTS-2009-Genzel.pdf
[3] A. Handler, An empirical study of semantic similarity in WordNet and Word2Vec, Columbia University, (2014).
[4] D. Joshi, J. Z. Wang and J. Li, The Story Picturing Engine|a system for automatic text illustration, ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP), 2(2006), 68-89.
[5] J. McCarty, Programs with common sense, Defense Technical Information Center, (1963).
[6] J. K. Poots and E. Bagheri, Automatic annotation of text with pictures, (in-press), IEEE IT Professional, (2016).
[7] S. Rose, D. Engel, N. Cramer and W. Cowley, Automatic keyword extraction from individual documents, Text Mining, (2010), 1-20.
[8] V. Uren, P. Cimiano, J. Iria, S. Handschuh, M. Vargas-Vera, E. Motta and F. Ciravegna, Semantic annotation for knowledge management:Requirements and a survey of the state of the art, Web Semantics:science, services and agents on the World Wide Web, 4(2006), 14-28.
[9] N. UzZaman, J. P. Bigham and J. F. Allen, Multimodal summarization of complex sentences, Proceedings of the 16th international conference on Intelligent user interfaces, 2(2004), 43-52.
[10] T. Veale, A. Conway and B. Collins, The challenges of cross-modal translation:English-toSign-Language translation in the Zardoz system, Machine Translation, 13(1998), 81-106.
[11] L. Zhao, K. Kipper, W. Schuler, C. Vogler, N. Badle and M. Palmer, A machine translation system from English to American Sign Language, Envisioning Machine Translation in the Information Future, (2000), 54-67.

show all references

This article has been cited by:

1. Jezia Zakraoui, Moutaz Saleh, Jihad Al Ja’am, Text-to-picture tools, systems, and approaches: a survey, Multimedia Tools and Applications, 2019, 10.1007/s11042-019-7541-4

Reader Comments

your name: * your email: *

Copyright Info: 2017, Kent Poots, et al., licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution Licese (http://creativecommons.org/licenses/by/4.0)

Download full text in PDF

Associated material

Metrics