Visual attention to different types of graphical representations in elementary school mathematics textbooks: An eye-movement-based study

Shumeng Ni; Zhujun Jiang; Fengkuang Chiang; Shumeng Ni; Zhujun Jiang; Fengkuang Chiang

doi:10.3934/steme.2025022

STEM Education

2025, Volume 5, Issue 3: 448-472. doi: 10.3934/steme.2025022

Previous Article Next Article

Research article Topical Sections

Visual attention to different types of graphical representations in elementary school mathematics textbooks: An eye-movement-based study

1.
Department of Linguistics, Shanghai Jiao Tong University, No. 800 Dongchuan Road, Minhang District, Shanghai, China; colormeblue@sjtu.edu.cn
2.
School of Education, Shanghai Jiao Tong University, No. 800 Dongchuan Road, Minhang District, Shanghai, China; zhujunjiang@sjtu.edu.cn, fkchiang@sjtu.edu.cn

Academic Editor: Kwesi Yaro

Received: 12 February 2025 Revised: 30 March 2025 Accepted: 02 April 2025 Published: 07 May 2025

Graphical representations play a crucial role in elementary school mathematics textbooks by conveying information visually and fulfilling specific educational functions. This study examined how seven students aged 9–11 years from Shanghai engaged with graphical elements in an elementary school mathematics textbook. Illustrations were classified into three types: decorative diagrams, guiding diagrams, and informational diagrams. By employing a combination of eye-tracking technology and retrospective interviews, the study analyzed and compared the attention allocation differences among these illustration types. The findings indicated that participants relied more on reading textual content than on illustrations. Moreover, they tended to pay less attention to guiding illustrations and more frequently referred to informational illustrations when reading text. This study identified several limitations in the design of mathematics textbooks and offered practical recommendations for improvement.
- eye-tracking movement,
- mathematics textbooks,
- graphical representations,
- visual attention patterns,
- improvement suggestions
Citation: Shumeng Ni, Zhujun Jiang, Fengkuang Chiang. Visual attention to different types of graphical representations in elementary school mathematics textbooks: An eye-movement-based study[J]. STEM Education, 2025, 5(3): 448-472. doi: 10.3934/steme.2025022

Related Papers:

Abstract

Graphical representations play a crucial role in elementary school mathematics textbooks by conveying information visually and fulfilling specific educational functions. This study examined how seven students aged 9–11 years from Shanghai engaged with graphical elements in an elementary school mathematics textbook. Illustrations were classified into three types: decorative diagrams, guiding diagrams, and informational diagrams. By employing a combination of eye-tracking technology and retrospective interviews, the study analyzed and compared the attention allocation differences among these illustration types. The findings indicated that participants relied more on reading textual content than on illustrations. Moreover, they tended to pay less attention to guiding illustrations and more frequently referred to informational illustrations when reading text. This study identified several limitations in the design of mathematics textbooks and offered practical recommendations for improvement.

References

[1]	Stefanel, A., Graph in Physics Education: From Representation to Conceptual Understanding, In Springer eBooks, 2019, 195‒231. https://doi.org/10.1007/978-3-030-04627-9_9
[2]	Yu, J., Research on the application of graphical representation method in high school physics teaching, Central China Normal University, 2018. Available from: http://www.cnki.net
[3]	de Jong, T., Ainsworth, S., Dobson, M., van der Hulst, A., Levonen, J., Reimann, P., et al., Learning with multiple representations, Oxford: Pergamon Press, 1998, 9‒40. https://doi.org/10.1007/978-1-4419-1428-6_485
[4]	Hegarty, M., Kozhevnikov, M., Types of visual–spatial representations and mathematical problem solving, Journal of Educational Psychology, 1987, 91(4): 684‒689. https://doi.org/10.1037/0022-0663.91.4.684 doi: 10.1037/0022-0663.91.4.684
[5]	Mayer, R. E., The Cambridge handbook of multimedia learning, 3rd ed, New York: Cambridge University Press, 2005. https://doi.org/10.1017/cbo9781139547369.002
[6]	Paivio, A., Mental representations: a dual coding approach, 2nd ed, New York: Oxford University Press, 1990. https://doi.org/10.1093/acprof: oso/9780195066661.003.0004
[7]	Sweller, J., Cognitive load theory: Recent theoretical advances, In J.L. Plass, R. Moreno, & R. Brunken (Eds.), Cognitive load theory, Cambridge University Press, 2010, 29‒47. https://doi.org/10.1017/CBO9780511844744.004
[8]	Ogren, M., Nystrom, M. and Jarodzka, H., There's more to the multimedia effect than meets the eye: is seeing pictures believing? Instructional Science, 2017, 263‒287. https://doi.org/10.1007/s11251-016-9397-6 doi: 10.1007/s11251-016-9397-6
[9]	Van Gog, T. and Scheiter, K., Eye tracking as a tool to study and enhance multimedia learning. Learning and Instruction, 2009, 20(2): 95‒99. https://doi.org/10.1016/j.learninstruc.2009.02.009 doi: 10.1016/j.learninstruc.2009.02.009
[10]	Levin, J.R., Anglin, G.J. and Carney, R.N, On empirically validating functions of pictures in prose, in The Psychology of Illustration, D.M. Willows and H.A. Houghton, New York: Springer, 1987, 51‒85. New York: Springer. https://doi.org/10.1007/978-1-4612-4674-9_2
[11]	Kress, G.R. and Van Leeuwen, T., Reading images: the grammar of visual designl, 3rd ed, Psychology Press, 1996. https://doi.org/10.4324/9780203619728
[12]	Wiley, J., Sarmento, D., Griffin, T.D. and Hinze, S.R., Biology textbook graphics and their impact on expectations of understanding. Discourse Processes, 2017, 54(5-6): 463‒478. https://doi.org/10.1080/0163853x.2017.1319655 doi: 10.1080/0163853x.2017.1319655
[13]	Feucht, F.C, Epistemic climate in elementary classrooms. In Cambridge University Press eBooks, 2010, 55‒93. https://doi.org/10.1017/cbo9780511691904.003
[14]	Chua, H.F., Boland, J.E. and Nisbett, R.E., Cultural variation in eye movements during scene perception. Proceedings of the National Academy of Sciences, 2005, 102(35): 12629‒12633. https://doi.org/10.1073/pnas.0506162102 doi: 10.1073/pnas.0506162102
[15]	Just, M.A., Carpenter, P.A., A theory of reading: From eye fixations to comprehension. Psychological Review, 1980, 87(4): 329–354. https://doi.org/10.1037/0033-295x.87.4.329 doi: 10.1037/0033-295x.87.4.329
[16]	Ke, F., Liu, R., Sokolikj, Z., Dahlstrom-Hakki, I. and Israel, M., Using eye-tracking in education: review of empirical research and technology, Educational Technology Research and Development, 2024, 72(3): 1383‒1418. https://doi.org/10.1007/s11423-024-10342-4 doi: 10.1007/s11423-024-10342-4
[17]	Hannus, M. and Hyona, J., Utilization of illustrations during learning of science textbook passages among low- and high-ability children. Contemporary Educational Psychology, 1999, 95‒123. https://doi.org/10.1006/ceps.1998.0987 doi: 10.1006/ceps.1998.0987
[18]	Jian, Y.C., Fourth graders' cognitive processes and learning strategies for reading illustrated biology texts: eye movement measurements. Reading Research Quarterly, 2016, 51(1): 93‒109. https://doi.org/10.1002/rrq.125 doi: 10.1002/rrq.125
[19]	Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H. and Van De Weijer, J., Eye Tracking: A comprehensive guide to methods and measures, 2015.
[20]	Posner, M., The attention system of the human brain, Annual Review of Neuroscience, 1990, 13(1): 25‒42. https://doi.org/10.1146/annurev.neuro.13.1.25 doi: 10.1146/annurev.neuro.13.1.25
[21]	Rayner, K., Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 1998, 124(3): 372‒422. https://doi.org/10.1037/0033-2909.124.3.372 doi: 10.1037/0033-2909.124.3.372
[22]	Lenzner, A., Schnotz, W. and Muller, A., The role of decorative pictures in learning. Instructional Science, 2013, 811‒831. https://doi.org/10.1007/s11251-012-9256-z doi: 10.1007/s11251-012-9256-z
[23]	Veenman, M.V.J., Van Hout-Wolters, B.H.A.M. and Afflerbach, P., Metacognition and learning: conceptual and methodological considerations. Metacognition and Learning, 2006, (1): 3‒14. https://doi.org/10.1007/s11409-006-6893-0 doi: 10.1007/s11409-006-6893-0
[24]	Mason, L., Pluchino, P. and Tornatora, M.C., Effects of picture labeling on science text processing and learning: evidence from eye movements. Reading Research Quarterly, 2013, 48(2): 199‒214. https://doi.org/10.1002/rrq.41 doi: 10.1002/rrq.41
[25]	Mason, L., Tornatora, M.C. and Pluchino, P., Integrative processing of verbal and graphical information during re-reading predicts learning from illustrated text: an eye movement study. Reading and Writing, 2015, 28: 851‒872. https://doi.org/10.1007/s11145-015-9552-5 doi: 10.1007/s11145-015-9552-5
[26]	Carney, R.N. and Levin, J.R., Pictorial Illustrations Still Improve Students' Learning from Text. Educational Psychology Review, 2002, 14: 5‒26. https://doi.org/10.1023/A:1013176309260 doi: 10.1023/A:1013176309260
[27]	Kalyuga, S., Expertise Reversal Effect and its Implications for Learner-Tailored Instruction. Educational Psychology Review, 2007, 19(4): 509‒539. https://doi.org/10.1007/s10648-007-9054-3 doi: 10.1007/s10648-007-9054-3
[28]	Kahneman, D. and Beatty, J., Pupil diameter and load on memory. Science, 1966, 154(3756): 1583‒1585. https://doi.org/10.1126/science.154.3756.1583 doi: 10.1126/science.154.3756.1583
[29]	McMaster, K.L., Espin, C.A. and van den Broek, P., Making connections: linking cognitive psychology and intervention research to improve the comprehension of struggling readers. Learning Disabilities Research & Practice, 2014, 29(1): 17‒24. https://doi.org/10.1111/ldrp.12026 doi: 10.1111/ldrp.12026
[30]	Wang, L., Research on the current situation and countermeasures of math reading for upper elementary school students, Phd thesis, 2018. Liaoning Normal University, Liaoning, China. Available from: http://www.cnki.net
[31]	Shah, D.A. and Madden, L.V., Nonparametric analysis of ordinal data in designed factorial experiments. Phytopathology, 2004, 94(1): 33‒43. https://doi.org/10.1094/phyto.2004.94.1.33 doi: 10.1094/phyto.2004.94.1.33
[32]	Fay, M.P. and Proschan, M.A., Wilcoxon-Mann-Whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules. Statistics Surveys, 2010, 4. https://doi.org/10.1214/09-ss051 doi: 10.1214/09-ss051
[33]	Pande, P. and Chandrasekharan, S., Expertise as Sensorimotor Tuning: Perceptual Navigation Patterns Mark Representational Competence in Science. Res Sci Educ, 2022, 52(2): 725‒747. https://doi.org/10.1007/s11165-020-09981-3 doi: 10.1007/s11165-020-09981-3
[34]	Ainley, M., Hidi, S. and Berndorff, D., Interest, learning, and the psychological processes that mediate their relationship. Journal of Educational Psychology, 2002, 94(3): 545‒561. https://doi.org/10.1037/0022-0663.94.3.545 doi: 10.1037/0022-0663.94.3.545
[35]	Hegarty, M. and Kozhevnikov, M., Types of visual–spatial representations and mathematical problem solving. Journal of Educational Psychology, 1999, 91(4): 684‒689. https://doi.org/10.1037/0022-0663.91.4.684 doi: 10.1037/0022-0663.91.4.684
[36]	Greer, D.L., Crutchfield, S.A. and Woods, K.L., Cognitive Theory of Multimedia Learning, Instructional Design Principles, and Students with Learning Disabilities in Computer-based and Online Learning Environments., Journal of Education, 2013, 193(2): 41‒50. https://doi.org/10.1177/002205741319300205 doi: 10.1177/002205741319300205
[37]	Mayer, R.E., The past, present, and future of the Cognitive Theory of Multimedia Learning. Educational Psychology Review, 2024, 36(1). https://doi.org/10.1007/s10648-023-09842-1 doi: 10.1007/s10648-023-09842-1
[38]	Rutten, N., Van Joolingen, W.R. and Van Der Veen, J.T., The learning effects of computer simulations in science education., Computers & Education, 2011, 58(1): 136‒153. https://doi.org/10.1016/j.compedu.2011.07.017 doi: 10.1016/j.compedu.2011.07.017
[39]	Nisbett, R.E., The geography of thought: how Asians and Westerners think differently--and why, In Free Press eBooks, 2003. https://doi.org/10.26522/brocked.v13i2.52
[40]	Erbas, A.K., Alacaci, C. and Bulut, M., A Comparison of Mathematics Textbooks from Turkey, Singapore, and the United States of America. Educational Sciences Theory & Practice, 2012, 12(3): 2324‒2329.
[41]	Sweller, J., Instructional design. Encyclopedia of evolutionary psychological science, 2021, 4159‒4163. Cham: Springer International Publishing. https://doi.org/10.1002/0470018860.s00673
[42]	Duchowski, A.T., Eye tracking Methodology: theory and practice, In Springer eBooks, 2003. https://doi.org/10.1007/978-1-4471-3750-4
[43]	Rim, N.W., Choe, K.W., Scrivner, C. and Berman, M.G., Introducing Point-of-Interest as an alternative to Area-of-Interest for fixation duration analysis. PLoS ONE, 2021, 16(5): e0250170. https://doi.org/10.1371/journal.pone.0250170 doi: 10.1371/journal.pone.0250170

Author's biography Shumeng Ni is an undergraduate student at Shanghai Jiao Tong University. Her research interests include STEM education, educational psychology, and psychology linguistics; Zhujun Jiang is a PhD candidate at the School of Education at Shanghai Jiao Tong University. Her research interests are STEM education and representation competence; Fengkuang Chiang is a professor at the School of Education at Shanghai Jiao Tong University. His research focused on STEM education, learning space, and integration of information technology and curriculum

Reader Comments

Your name:*

Email:*
© 2025 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)