A meta-analysis of the effect of modelling activities on learning outcomes in mathematics

1.   Introduction

Mathematical modelling is introduced to overcome complex problems ^[22,29]. Complex problems were assessed in PISA to gain information on students' skills to acquire and apply knowledge while dealing with non-transparent problem situations ^[3,25]. It is important to identify possible areas of development and provide learning environments that improve students' complex problem-solving skills to prepare them for the challenges of the 21st century. This should allow students to improve their knowledge through new representations and imagination and, therefore, to learn ^[8]. In a mathematics modelling classroom, students will be able to comprehend and solve practical problems ^[27] since the modelling task is related to real context issues, from health, economic, social, and scientific fields. Modelling competency refers to skills simplifying, mathematising, computing, interpreting, and validating ^[20]. Modelling competency is also defined as "the ability to use a wide range of tools to solve complex real-world problems and to work collaboratively with people". Modelling competency can aid education in science, technology, engineering, and mathematics (STEM). Critical thinking, issue-posing, problem-solving, collaborating, and communicating are all abilities that have long been emphasised in STEM education ^[15]. Therefore, the appropriate use of mathematics modelling can help students develop competencies in their understanding of the problem and the realisation of the different stages of the modelling process ^[20]. In addition, the modelling process enables students to improve their modelling competencies, namely understanding and simplifying, organising problems, mathematising, working mathematically, interpreting and verifying solutions, and presenting solutions ^[18].

The superiority of mathematical modelling activities compared with conventional teaching in increasing students' mathematical communication and problem-solving abilities has been widely reported. Model-eliciting activities (MEAs) are mathematical learning models that use mathematical modelling to analyse, explain, and express mathematical concepts included in a challenging presentation ^[22]. MEAs are purposefully tailored to incorporate the mathematics process in the produced model. This serves as a major difference from mathematical modelling tasks within realistic and educational perspectives because an MEA is designed with the goal of representing the mathematical process through task engagement ^[19]. Mathematical modelling perspective-centred model-eliciting activities provide a learning environment that fosters the development of many essential skills outlined in the curriculum ^[1]. MEAs employ mathematical modelling to analyse, explain, and express challenging mathematical concepts ^[22]. In addition, MEAs are a curricular approach that has been presented to help students develop mathematical creativity, though that was not their original intent. MEAs have also been shown to provide valuable insights into a student's conceptual mathematical understanding. In addition, MEAs make an important contribution to the educational context. First, MEAs implemented with Cabri 3D improve students' reasoning skills through the application of learning materials and workbooks. Research has shown the positive impact of MEAs, highlighting a significant relationship between students' performance in the classroom with their reasoning abilities ^[34]. Chamberlin and Moon ^[10] described MEAs as mathematical tasks that require students to create models and solve complex problems. They provide instructors with invaluable insights in the form of assessment data, which enables them the opportunity to analyse cognitive processes carefully during mathematical problem-solving episodes ^[13,25].

To better understand how MEAs can be used in education, a systematic literature review (SLR) and meta-analysis of research published in the last 5 years were conducted. Synthesising the results of previous studies is crucial for understanding the effect of modelling activities on mathematics education. Abassian et al. ^[1] performed the most recent literature review on modelling perspectives, analysing the existing research in mathematics education settings. However, current research does not concentrate specifically on modelling activities, and the inconsistency between the studies in question casts doubt on how modelling activities affect learning outcomes in mathematics. Such inconsistency arises from their lack of focus on modelling activities, resulting in conflicting findings regarding their influence on learning outcomes in mathematics. This study aims to address this gap by specifically investigating the effects of modelling activities and providing a more detailed understanding of their role in improving mathematics learning. Previous research has concentrated primarily on problem-solving skills among secondary school students. For example, Armutcu and Bal ^[4] examined the influence of mathematical modelling activities on students' mathematical modelling abilities. In the context of STEM education, students who participated in mathematical modelling activities integrated into STEM education demonstrated the ability to adopt different interdisciplinary perspectives, exhibited positive developments in their cognitive abilities, adapted to group work more efficiently, and displayed heightened interest in engineering and technology. Similarly, Hartati et al. ^[17] also found that the application of modelling activities to mathematical problem-solving skills and mathematical reflective thinking skills is relatively effective. The way in which the learning environment is presented can influence students' attitudes and beliefs about themselves and the lessons, which in turn can have a positive impact on their mathematical skills and increase their success. However, their studies were limited in scope in terms of the topics covered and the levels of education involved. Finally, the current SLR can contribute to the literature on the impact of modelling activities on learning outcomes in mathematics. We propose the following research questions:

1. What is the overall impact of MEAs on cognitive skills?

2. What is the overall impact of MEAs on problem-solving skills?

3. What is the overall impact of MEAs on learning outcomes in mathematics?

2.   Literature review

2.1.   Model-eliciting activity (MEA)

Mathematical learning models such as MEAs employ mathematical modelling to analyse, explain, and express challenging mathematical concepts ^[22]. MEAs are defined as client-driven, open-ended problem-solving exercises in which students work in small groups to construct and deliver functional solutions. MEAs are problem-solving exercises that challenge students to communicate their existing methods of thinking in formats that are revised several times to elicit a model. As posited by Lesh and Doerr ^[21], MEAs encourage students to devise distinctive mathematical models to address complex problems. The six core principles of MEA, as outlined by Handajani and Pratiwi ^[22], are as follows: model construction, reality, self-assessment, construct documentation, shareability and reusability, and effective prototype.

The implementation of MEAs has been observed to have a positive impact on student engagement. The fostering of mutual respect among students within the context of MEA has been observed to engender a greater willingness to express one's views ^[8]. Previous research indicated that instructors do not neglect students who are underperforming academically, as every student is afforded the opportunity to engage in activities ^[8]. MEAs allow students to interpret, develop, and identify solutions in ways that circumvent the limitations of the "success stereotype" by prioritising processes over more rigid techniques ^[22]. Furthermore, the structure of MEAs enables problem-solvers to concentrate on the process of finding a solution rather than seeking a single answer through established mathematical techniques ^[21]. In brief, the modelling cycle is the process through which students define, manipulate, predict, and validate their mathematical creations before adapting, modifying, and/or refining their knowledge and concepts within MEAs ^[21].

Furthermore, MEAs have been demonstrated to be a valuable addition to the educational landscape. First, the utilisation of MEAs with Cabri 3D improves students' reasoning abilities through the implementation of learning materials and workbooks as an application tool. Previous research has demonstrated a significant relationship between students' performance in the classroom and their reasoning abilities ^[34]. Moreover, MEAs have been applied by Bartholomew and colleagues ^[6]; 706 students in a suburban area in the western United States were analysed, and it was found that MEAs demonstrated an acceptable level of validity, reliability, and feasibility, as well as providing access to modelling questions ^[8].

2.2.   The models and modelling perspective (MMP)

MMP, also known as contextual modelling ^[19], is seen as a problem-solving, learning, and teaching perspective that involves a continuous cycle of express-test-revise-refine mathematical thinking. In addition, MMP also provides a methodology for interpreting real-world scenarios and developing formal mathematical knowledge in line with students' understanding ^[11]. The initial stage of the MMP process is a transition from the tangible, real-world context to the abstract, mathematical domain. This is achieved by leveraging students' existing mathematical knowledge as a foundation for modelling, which includes not only computational techniques but also learning theories ^[1]. It is evident that MMP goes beyond the conventional approach of imparting mathematical knowledge to learners. Moreover, MMP considers a mathematical model as a conceptual instrument of a mathematical system that emerges from a specific real-world context ^[21]. In contrast to the conventional approach of viewing mathematical models as geometric figures that relate to a specific real-world scenario, MMP views mathematical models as systems that encompass different concepts that represent different aspects of related components ^[1]. In other words, the concept of MMP represents a novel approach to integrating real-world contexts into the learning and teaching of mathematical problem-solving. MMP draws on the work of Vygotsky, Piaget, and Dienes, as well as influences from the tradition of American pragmatism represented by Mead, Peirce, and Dewey. Accordingly, constructivism forms the theoretical basis of MMP ^[14].

The application of MMP to the teaching and learning of mathematics is crucial, as it prepares students to be mentally active in the modelling process. Students will use their internal conceptual systems to organise, understand, and make connections between events, experiences, or problems ^[14]. Students will also actively design their models in accordance with the basic concepts of constructivism. Adapting MMP to student learning will also facilitate communication between peers and teachers through project-based learning or problem-based learning ^[4]. Students will actively participate in classroom activities by discussing the learning and modelling process. The complexity of the MMP supports students in developing their modelling competencies and may connect students to work in real-life contexts.

MMP enables students to develop the ability to solve simulations of realistic, engaging mathematical problems. This is achieved by encouraging students to engage in mathematical thinking, that involves the creation and interpretation of situations that require the description, explanation, communication, and computation of data, as well as the execution of procedures and deductive reasoning. In addition, the mathematical modelling process emphasises group work through a variety of activities. Sharing ideas for mathematical problem-solving through social interaction with peers is a method that has been used in numerous research studies ^[9]. Students may employ a variety of strategies to solve the deliberately developed open-ended problem. Another method for developing the theoretical approach is the utilisation of MEAs ^[14]. Students must intuitively grasp the mathematical principles inherent in a real-world situation and develop meaningful models within a limited time frame. In addition, the MMP is structured around a series of activities that require students to understand the real-world problem in the first phase. Students are then asked to structure the situation model, mathematise the situation to develop a mathematical model, and collaborate on mathematical models to generate mathematical results that are then validated in the real-world context and presented as a solution to the real-world problem.

3.   Methodology

3.1.   Research design

This section of the study deals with the effects of model-eliciting activities in an educational context. To identify and select studies for coding and analysis, an SLR and a meta-analysis were conducted according to PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. A systematic review is a scientific process with explicit rules to ensure completeness, impartiality, transparency, and accountability ^[7,16]. It is a methodological process of collecting and collating published empirical studies that meet the required quality standards and are selected according to systematic criteria. This is done to reduce the potential bias in research and make the process transparent ^[35]. Notwithstanding the criticism that systematic reviews limit the scope of possible outcomes ^[10,36], they offer several potential benefits, including the convergence of quantitative and qualitative research findings. The collection of secondary data for analysis was conducted using repeatable analytical methods. This approach is methodologically necessary as other commonly used analytical methods only cover either qualitative or quantitative paradigms ^[9]. A meta-analysis is a statistical technique that estimates effect sizes for a population that are not directly observable by synthesising quantitative results from primary studies. In contrast, a descriptive review study compiles, codes, and analyses numerical data that reflect the frequency analysis of the body of research and concentrates on overall study characteristics and methods, including authors, years of publication, research methods, school level, themes, topics, task types, and direction of study findings. The PRISMA guidelines include a 27-item checklist and a four-phase flowchart that describes the key points for transparency in the conduct of literature reviews ^[35]. The objective of a systematic review is to provide a summary of the current evidence relevant to the research question. Essentially, a systematic review comprises five primary phases: identification, screening, eligibility determination, application of inclusion and exclusion criteria, and data analysis.

3.1.1.   Identification phase

In the identification phase, the following electronic databases were consulted to investigate the impact of model-eliciting activities: ProQuest, ScienceDirect, Scopus, and SpringerLink. Although Science Direct is a full-text scientific database that is part of SciVerse and provided by the medical and scientific publisher Elsevier ^[26], it contains research papers on mathematical modelling. However, only a few of these were available for assessment. ERIC includes a wide range of academic articles, reports, and educational resources, making it an asset for researchers in the education field. In addition, ProQuest is known for its extensive collection of dissertations, theses, and scholarly publications, providing researchers with a diverse selection of resources to explore in the field of education ^[24]. In addition, SpringerLink is a comprehensive online platform that provides easy access to more than 8 million resources and is the world's most comprehensive online collection of STM books, journals, reference works, protocols, and databases. It is based on the latest technologies and has sophisticated search and retrieval tools that deliver results with the speed, accuracy, and convenience that researchers need today. To ensure comprehensive coverage, the search was extended to Google Scholar to ensure that all pertinent articles had been identified. The literature search was conducted between September and November 2024. The use of Google Scholar as a search engine is not subject to the same restrictions on search terms and resulted in 2930 results, which were subsequently sorted by relevance. As recommended by Haddaway et al. ^[16], the first 200–300 results from Google Scholar were examined to identify any overlooked literature. The abstracts of the first 300 articles were reviewed to ascertain their relevance to the research question. The following search terms were employed to search each database: "model-eliciting activities", "modelling task", "modelling problem", "mathematical modelling task", "mathematical modelling problem", "mathematical modelling competencies", or "real-world problems^* and in terms of education". All searches were conducted using article abstracts, and the search criteria were matched against the screening criteria (see Figure 1).

Figure 1.  PRISMA flowchart.

DownLoad: Full-Size Img PowerPoint

3.1.2.   Screening phase

In the subsequent screening phase, the previously defined selection criteria were applied in order to exclude all irrelevant research after all results from the selected sources had been recorded. For the selection of studies to be included, the following criteria were applied sequentially to the article abstracts: (1) the study must have been published between 2019 and 2024; (2) the study must have been published in a scholarly journal; (3) the study must have employed model-eliciting activities as a teaching approach; (4) the study must be based on a quantitative (quasi-experimental) design or a mixed method; (5) the extracted data must be consistent with the focus and research questions of the current study.

3.1.3.   Eligibility

It is recommended that studies selected for inclusion in a review be subjected to a predefined criterion before undergoing a more thorough and detailed screening. Consequently, the full-text articles considered for inclusion in this review had to have undergone a peer-review process and been published in a scholarly journal between 2019 and 2024 (trade journals, magazines, and newspapers were excluded). In addition, the study had to answer at least one of the study's research questions. The studies that appeared to be eligible for inclusion were subjected to the screening process. In the end, a total of 38 full-text articles were screened for eligibility. Based on the keywords, it was determined that the articles retrieved from the database were related to the effect of model-eliciting activities with a focus on the educational context.

3.1.4.   Inclusion and exclusion criteria

The review considered studies that explored any aspect of the impact of model-eliciting activities in an educational context. The population was not predefined, provided that participants were engaged in some form of academic activity. The themes of achievement, outcome, and creativity were addressed. Of particular interest were the criteria relating to the methodology of the study. Only studies employing a specific quantitative design (experimental) were included in the review. In addition, studies with a mixed methods design that included a detailed quantitative component were also considered. In addition, this review considered studies that explored learners' outcomes (cognitive and affective outcomes) when they were related to the effect of model-eliciting activities, as previously described. This implies that the review excluded articles that focused exclusively on the pre-test and post-test phases of the experimental design, without clarifying the control and experimental groups. In addition, studies were excluded that had methodological flaws and those that were conducted for learners without demonstrating an impact on the study variable. Ultimately, 20 articles were selected that met the specified selection criteria.

The meta-analyses were performed with the R software, version 4.3.0. Two separate meta-analyses were conducted: one focused on academic achievement and the other on additional outcomes. Effect sizes (ES) were calculated as standardised mean difference (SMD) with a 95% confidence interval. To interpret the results, effect sizes between 0.2 and 0.5 were categorised as small, between 0.5 and 0.8 as moderate, and above 0.8 as large. Given the expected heterogeneity among the included studies and the differences in sample characteristics and measurement methods, a random effects model was used.

4.   Results

4.1.   Study characteristics

After applying the eligibility criteria, 20 papers were selected for inclusion in the systematic analysis. The primary eligibility criteria were methodology and descriptive statistics regarding effect sizes. In the present study, six research questions were asked to review the selected articles. The results of the meta-analysis, derived from 20 studies, examine the effects of modelling activities on educational outcomes. A total sample size of 3047 (experimental, N = 1543; control, N = 1504) was identified in these studies. Table 1 provides an overview of the selected studies that had to fulfil the inclusion criteria. The table includes information on the countries studied, the research designs employed, the school level for each learner, the themes currently investigated, the type of tasks used in previous studies, and the outcomes of the effect size analysis.

Table 1.  Study characteristics.

No.	Authors/year	Country	School level	Research design	Themes	Types of tasks	Outcomes
1	Armutcu & Bal (2023)	Turkey	Secondary school	Mixed method (quasi-experimental)	STEM	Procedural task	Mathematical modeling skill: [EG: 36.66 ± 8.81], N = 33; CG [15.06 ± 6.06], N = 33
2	Fidan & Tuncel (2019)	Turkey	Junior High School	Quasi-experimental	Force and energy	Problem-solving task	Learning achievement: [EG1: 29.23 ± 4.56], N = 30; EG2 [25.96 ± 4.67], N = 31; CG [22.73 ± 5.42]; N = 30 Attitude: [EG1: 122.40 ± 17.44], N = 30; EG2 [106.23 ± 22], N = 31; CG [98.60 ± 14.07], N = 30
3	Hartati et al. (2020)	Indonesia	Secondary school	Quasi-experimental	Mathematics	Problem-solving task	Problem-solving ability: [EG: 68.33 ± 13.28], N = 30; CG [23.07 ± 14.58], N = 30 Reflective thinking ability: [EG: 82.80 ± 15.31], N = 30; CG [72.80 ± 12.85], N = 30
4	Karabörk & Durmus (2020)	Turkey	Secondary school	Mixed method (quasi-experimental)	Mathematics	Problem-solving task	Achievement: [EG: 63.98 ± 17.81], N = 23; CG [52.98 ± 19.22], N = 24
5	Khalil et al. (2023)	United Arab Emirates	High school	Quasi-experimental	Physics	Inquiry-based task	Fluency: [EG: 1.202 ± 1.003], N = 48; CG: [0.861 ± 1.027], N = 46 Elaboration: [EG: 1.312 ± 0.839], N = 48; CG: [1.072 ± 0.989], N = 46 Flexibility: [1.586 ± 0.2], N = 48; CG: [1.288 ± 1.079], N = 46 Originality: [EG: 1.185 ± 1.054], N = 48; CG: [1.062 ± 1.009], N = 46
6	Lee et al. (2021)	Singapore	Secondary school	Quasi-experimental	Gradient of linear graphs	Problem-solving task	Knowledge: [EG: 7.68 ± 2.12], N = 23; CG [6.48 ± 2.91], N = 18 Understanding: [EG: 5.94 ± 3.89], N = 23; CG [1.86 ± 2.05], N = 18 Near transfer: [EG: 3.91 ± 3.00], N = 23; CG [1.94 ± 3.04], N = 18 Far transfer: [EG: 2.83 ± 2.39], N = 23; CG [1.53 ± 1.74], N = 18
7	Li & Zulnaidi (2019)	Malaysia	Secondary school	Quasi-experimental	Quadratic equation	Problem-solving task	Reasoning competency: [EG: 40.90 ± 5.17], N = 30; CG [32.47 ± 7.27], N = 30
8	Nguyen & Santagata (2021)	California	Middle school	Quasi-experimental	Science	Procedural task	Element: [EG: 3.70 ± 1.078], N = 60; CG [3.525 ± 1.12], N = 59 Evidence: [EG: 2.050 ± 1.199], N = 60; CG [1.475 ± 1.237], N = 59 Causal coherence: [EG: 3.00 ± 1.529], N = 60; CG [2.339 ± 1.434], N = 59
9	Sembiring & Simanjorang (2019)	Indonesia	Secondary school	Quasi-experimental	System of linear equations of two variables	Problem-solving task	Problem-solving ability: [EG: 81 ± 7.61], N=21; CG [70 ± 8.92], N=22 Ability (Man): [77.00 ± 6.00], N = 21; CG: [66.00 ± 8.54], N = 22 Ability (Women): [EG: 88.00 ± 4.41], N = 21; CG: [74.00 ± 7.17], N = 22
10	Sumarmo et al. (2019)	Indonesia	Secondary school	Quasi-experimental	Mathematics	Problem-solving task	Mathematical reasoning ability: [EG: 23.33 ± 2.66], N = 36; CG: [12.94 ± 3.31], N = 36 Self-regulated learning: [EG: 106 ± 8.14], N = 36; CG: [98.39 ± 8.20], N = 36
11	Wakhata et al. (2022)	Uganda	Secondary school	Mixed method (quasi-experimental)	Linear programming	Problem-solving task	Achievement: [EG: 65.9 ± 14.49], N = 285; CG [47.12 ± 16.58], N = 317
12	Susanti & Juandi (2021)	Indonesia	High school	Quasi-experimental	Trigonometry	Essay task	Critical thinking skill: [EG: 0.74 ± 0.11], N = 35; CG [0.66 ± 0.17], N = 32
13	Nurochmah & Kharisudin (2023)	Indonesia	Junior High School	Mixed method (quasi-experimental)	Straight-line equation	Problem-solving task	Problem-solving ability: [EG: 74.444 ± 10.916], N = 27; CG [66.964 ± 13.569], N = 28
14	Alt & Kapshuk (2022)	Israel	Higher education	Mixed method (quasi-experimental)	Digital concept mapping	Problem-solving task	Absolutist: [EG: 0.474 ± 0.503], N = 59; CG [0.438 ± 0.499], N = 73 Multiplist: [EG: 0.372 ± 0.487], N = 59; CG [0.383 ± 0.489], N = 73; Evaluatist: [EG: 0.153 ± 0.362], N = 59; CG [0.141 ± 0.350], N = 73
15	Krawitz et al., 2022	Germany & Taiwan	High-track schools	Quasi-experimental	Pythagorean Theorem	Paper-and-pencil modelling task	Construction of real-world model: [EG: 0.518 ± 0.284], N = 495; CG [0.512 ± 0.279], N = 495
16	Huffman & Mentzer (2019)	USA	High school	Quasi-experimental	End-of-course design challenge	Design challenge task	Design performance: [EG: 45.18 ± 11.74], N = 142; CG [42.56 ± 9.47], N = 124
17	Kharisudin & Cahyati (2020)	Indonesia	Secondary school	Mixed method (quasi-experimental)	counting rules, permutations, and combinations materials	Modelling task	Problem-solving ability: [EG: 71.72 ± 16.3], N = 27; CG [64.05 ± 19.9], N = 25
18	Ayuningtyas et al. (2023)	Indonesia	Junior High School	Quasi-experimental	Flat wake problems	-	Creative thinking ability: [EG: 26.56 ± 3.95], N = 25; CG: 19.67 ± 6.16], N = 25
19	Hung & Tsai (2020)	Taiwan	Secondary school	Quasi-experimental	Heat and specific heat	Modelling task	Performance: [EG1: 16.160 ± 4.836], N = 25; EG2 [15.786 ± 5.527], N = 28; CG [13.667 ± 4.690], N = 27
20	Mahmudin et al. (2020)	Indonesia	Senior High School	Quasi-experimental	Trigonometry	Mathematical creative thinking ability task	Mathematical creative thinking ability: [35.91 ± 6.14], N = 30; CG [29.11 ± 5.87], N = 30 Mathematical self-efficacy: [EG: 96.14 ± 5.37], N = 30; CG [75.66 ± 3.96], N = 30

---

 | Show Table

DownLoad: CSV

4.2.   Research learning outcomes

4.2.1.   Evaluation of methodological quality

The PEDro scale, with a maximum score of 10 points, was employed to assess the quality of the selected articles. The criteria for quality assessment were as follows: good quality (≥6 points), fair quality (4–5 points), and poor quality (≤3 points) ^[36]. Table 2 presents the assessment of the methodological quality of the included studies, as assessed using the PEDro scale. The PEDro scores of the included studies ranged from 6 to 7, with a mean PEDro score of 7 out of 10. The quality of the studies was of an exemplary standard. However, the primary limitation was the lack of random allocation of participants to groups. It should be noted that none of the studies involved concealed allocation or blinding of subjects, patients, or therapists.

Table 2.  Physiotherapy evidence database (PEDro) scale ratings for the included studies.

No	Study	Year	Scale items											Total (out of 10)
			1	2	3	4	5	6	7	8	9	10	11
1	Wakhata et al.	2022	√	√	X	√	X	X	X	√	√	√	√	6
2	Karabörk & Durmus	2020	√	X	X	√	X	X	X	√	√	√	√	6
3	Hartati et al.	2020	√	X	X	√	X	X	X	√	√	√	√	6
4	Nguyen & Santagata	2021	√	√	X	√	X	X	X	√	√	√	√	7
5	Lee et al.	2021	√	√	X	√	X	X	X	√	√	√	√	7
6	Khalil et al.	2023	√	√	X	√	X	X	X	√	√	√	√	7
7	Sembiring & Simanjorang	2019	√	X	X	√	X	X	X	√	√	√	√	6
8	Armutcu & Bal	2023	√	√	X	√	X	X	X	√	√	√	√	7
9	Sumarmo et al.	2019	√	X	X	√	X	X	X	√	√	√	√	6
10	Fidan & Tuncel	2019	√	√	X	√	X	X	X	√	√	√	√	7
11	Li & Zulnaidi	2019	√	√	X	√	X	X	X	√	√	√	√	7
12	Susanti & Juandi	2021	√	√	X	√	X	X	X	√	√	√	√	7
13	Nurochmah & Kharisudin	2023	√	√	X	√	X	X	X	√	√	√	√	7
14	Alt & Kapshuk	2022	√	√	X	√	X	X	X	√	√	√	√	7
15	Krawitz et al.,	2022	√	√	X	√	X	X	X	√	√	√	√	7
16	Mahmudin et al.	2020	√	√	X	√	X	X	X	√	√	√	√	7
17	Huffman & Mentzer	2019	√	√	X	√	X	X	X	√	√	√	√	7
18	Kharisudin & Cahyati	2020	√	√	X	√	X	X	X	√	√	√	√	7
19	Ayuningtyas et al.	2023	√	√	X	√	X	X	X	√	√	√	√	7
20	Hung & Tsai	2020	√	√	X	√	X	X	X	√	√	√	√	7
Note: Item 1: Eligibility criteria specified (not included in total score). Item 2: Random allocation of participants to groups. Item 3: Concealed allocation. Item 4: Similarity of groups at baseline. Item 5: Blinding of all subjects. Item 6: Blinding of all therapists. Item 7: Blinding of all assessors. Item 8: More than 85% of subjects had outcome measures obtained. Item 9: At least one key outcome was analysed. Item 10: At least one key outcome compared. Item 11: At least one key outcome: point and variability measures provided. The detailed explanation for the PEDro scale can be accessed at https://pedro.org.au/english/resources/pedro-scale/ (access for this review: September 3, 2024).

---

 | Show Table

DownLoad: CSV

4.3.   Overall effect of modelling activities on thinking skills

Our primary research objective is to determine the influence of modelling activities on student thinking skills (Figure 2). Moderate and positive effects were observed [effect size (ES) = 0.78, 95% confidence interval (CI) [0.34, 1.23], p < 0.01]. This suggests that students in the treatment groups experienced a significantly greater improvement in thinking skills than those in the control groups.

Figure 2.  Forest plot of the modelling activities' impact on thinking skills.

DownLoad: Full-Size Img PowerPoint

4.4.   Overall effect of modelling activity on problem-solving skills

Our secondary research objective is to determine the influence of modelling activities on students' problem-solving skills, shown in Figure 3. Large and positive effects of modelling activities were observed in problem-solving skills (ES = 0.97, 95% CI [0.46, 1.49], p < 0.01). This suggests that students in the treatment groups experienced significantly greater enhancement in problem-solving skills than those in the control groups.

Figure 3.  Forest plot of modelling activities' impact on problem-solving skills.

DownLoad: Full-Size Img PowerPoint

4.5.   Overall effect of modelling activities on achievements

Our third research objective is to determine how modelling activity influences achievements. Significant and moderate effects of modelling activities were observed in these areas (ES = 0.70, 95% CI [0.38, 1.02], p < 0.01) (Figure 4). This suggests that students in the treatment groups experienced a significantly greater improvement in these achievements than those in the control groups.

Figure 4.  Forest plot of modelling activities' impact on achievements.

DownLoad: Full-Size Img PowerPoint

4.6.   Publication bias on thinking skills

Our analysis showed that the funnel plot displayed a consistent distribution, indicating little evidence of publication bias regarding thinking skills (Figure 5). Moreover, the lack of significance in Egger's test result (b = 3.65, df = 14, p = 0.0001) further evidenced no publication bias. These results emphasise the reliability and strength of our meta-analysis outcomes, indicating that selective publication practices did not unfairly influence our conclusions. The Egger tests' results validated the balanced distribution observed in the funnel plots.

Figure 5.  Funnel plot in thinking skills.

DownLoad: Full-Size Img PowerPoint

4.7.   Publication bias on problem-solving skills

Our funnel plot exhibited a balanced distribution, indicating that there is no significant evidence of publication bias in relation to problem-solving skills (Figure 6). The non-significant result of the Egger test (b = 10.12, df = 14, p = 0.0007) provided further support against the presence of publication bias. These findings underscore the reliability and robustness of the results of our meta-analysis and indicate that our conclusions were not unfairly influenced by selective publication practices. The results of the Egger tests confirmed the symmetrical distribution observed in the funnel plots.

Figure 6.  Funnel plot in problem-solving skills.

DownLoad: Full-Size Img PowerPoint

4.8.   Publication bias on achievements

Our funnel plot exhibited a balanced distribution, indicating that there is no significant evidence of publication bias concerning achievements (Figure 7). The non-significant result of the Egger test (b = 4.26, df = 6, p = 0.0001) provided further support against the presence of publication bias. These findings underscore the reliability and robustness of the results of our meta-analysis and indicate that our conclusions were not unfairly influenced by selective publication practices. The results of the Egger tests confirmed the symmetrical distribution observed in the funnel plots.

Figure 7.  Funnel plot in achievements.

DownLoad: Full-Size Img PowerPoint

5.   Discussion

Our results show that mathematical modelling activities are a very effective approach to improve thinking skills in mathematics learning and show a medium effect size. These findings are consistent with previous research ^[13,24,30], which has also shown that modelling activities significantly improve students' problem-solving skills and reflective thinking in mathematics compared to traditional teaching methods. An important reason for the strong and significant results of our study could be that the modelling activities are designed to immerse students in real-life problem-solving scenarios and require them to apply mathematical concepts and reasoning while working collaboratively. This approach not only reinforces an understanding of mathematical principles but also fosters the development of critical thinking skills that are crucial for academic success and future career aspirations. Incorporating modelling activities into an engineering curriculum resulted in remarkable improvements in students' critical thinking skills, as measured by pre- and post-test assessments. Astuti et al. ^[5] found that modelling activities promote fluency, flexibility, and originality in solving mathematical problems, key components of creative thinking. In addition, Wang et al. ^[28] found that students with stronger mathematical modelling competencies also exhibited higher levels of creativity. As students engage with the different stages of mathematical modeling, they are likely to generate diverse ideas and explore various problem-solving strategies, which helps cultivate their divergent thinking skills.

Our results show that mathematical modelling activities are an extremely effective method for improving problem-solving skills, with a strong effect size of 0.97. These results are consistent with prior research by Ramadhani ^[23]. These activities not only provide a systematic approach to understanding complex mathematical concepts but also facilitate the application of mathematical knowledge to real-world scenarios. The link between mathematical modelling and problem-solving is well-established in educational research and highlights the importance of modelling as a valuable pedagogical tool. A major advantage of mathematical modelling is its alignment with Polya's stages of problem-solving: understanding the problem, creating a plan, executing the plan, and evaluating the solution. This approach mirrors the steps of modelling, where students first analyse the problem, construct a model, solve it, and interpret the results. By incorporating these steps, mathematical modelling deepens students' understanding of mathematical principles and strengthens their critical thinking skills, both of which are essential for effective problem-solving ^[14]. For example, one study has shown that integrating discovery learning models, which include mathematical modelling, leads to significant improvements in students' problem-solving skills ^[23].

The significant positive impact of mathematical modelling activities on students' mathematical achievement is consistent with previous studies ^{[26,29,31,34]}. This suggests that although modelling activities generally improve mathematical performance, their effect may vary depending on the context and the students involved. One possible explanation for the strong and significant results found here is the ability of modelling activities to link abstract mathematical concepts to real-world applications, thus increasing student engagement and understanding. Dede et al. ^[12] highlighted that mathematical modelling tasks embody a range of pedagogical values, which can have a profound influence on students' learning experiences and outcomes. By engaging students in modelling tasks, educators can promote a deeper understanding of mathematical concepts, as students apply these principles in practical contexts. This connection between theory and practise is vital for developing positive attitudes toward mathematics, which research shows is associated with better academic performance ^[2,32]. Furthermore, the emphasis on modelling in modern mathematics education is in line with global educational standards that advocate the integration of modelling skills into curricula. Gravemeijer et al. ^[15] emphasised that mathematical modelling is a key element in many international educational standards that support a more holistic approach to mathematics education. This alignment with global standards not only highlights the importance of modelling but also encourages educators to incorporate these practises into their teaching strategies.

These results support the claim of the MMP model that engaging students in modelling activities helps them to interpret real-life situations and build formal mathematical knowledge. In particular, the significant impact on problem-solving skills challenges traditional educational theories that emphasise the isolated teaching of abstract mathematical concepts. Instead, the findings suggest that students benefit more from an approach that incorporates real-world problems, leading to a deeper understanding of mathematical principles. Furthermore, the findings challenge existing educational theories that may undervalue the role of modelling activities in promoting cognitive development and academic achievement. By highlighting the significant benefits of modelling for both problem-solving and overall achievement, the study advocates for a shift towards a more practical and integrated method of teaching mathematics, as envisioned by the MMP model. This reinforces the notion that student understanding and performance in mathematics can be improved when learning is based on real-world applications.

Another important factor contributing to the impact of MEAs is the fact that different educational systems, cultural influences, socio-economic status, and country-specific contexts can significantly affect their effectiveness in improving learning outcomes in mathematics ^[33]. In educational systems that emphasise inquiry-based or active learning, modelling activities can produce stronger results and lead to improvements in cognitive and problem-solving skills, and overall achievement. Cultural factors, such as attitudes toward mathematics and collaborative learning, may also affect how effectively students engage in modelling activities, which could improve outcomes in cognitive and problem-solving skills. Socio-economic status plays a role as it affects access to resources and support. Students from higher socio-economic backgrounds often benefit more from these activities as they have better access to technology and educational support. In addition, the country-specific context, including curriculum design and teacher training, may influence how widely and effectively modelling activities are implemented, which in turn affects their impact on learning outcomes. Based on our findings, which show significant moderate effects on cognitive skills, significant effects on problem-solving skills, and moderate effects on achievement, these factors collectively help to explain the variability in the success of modelling activities in different contexts.

5.1.   Conclusions and implications

The findings of this study support the notion that modelling activities have a positive influence on the acquisition of mathematical knowledge. Furthermore, there is a significant positive correlation between the utilisation of modelling activities and the enhancement of cognitive skills, including thinking skills, problem-solving, and overall academic achievement. It is recommended that teachers identify modelling activities that align with the mathematical content of their existing curriculum to enhance the modelling task, problem-solving task, or even the word task. This can be achieved by providing appropriate learning approaches and an appropriate learning environment. The findings presented in this study also support the use of modelling activities to improve students' assessment performance in design challenges. Teachers can encourage students to achieve robust targets. In addition, teachers should adopt effective methods in mathematical modelling lessons. For example, teachers should use modelling activities as structured practise to promote students' understanding of the role of mathematical concepts.

The remarkable impact of modelling activities on cognitive skills, problem-solving skills, and academic achievement underlines their value for mathematics education. However, challenges such as inadequate teacher training and inflexible curriculum structures can hinder successful implementation. Many educators lack the necessary knowledge and experience to effectively incorporate modelling activities. However, this can be remedied through professional development and continuous support. In addition, existing curricula often emphasise abstract concepts and standardised tests, which can make it difficult to integrate real-world applications. This challenge can be overcome by redesigning curricula to emphasise problem-solving and real-world scenarios, so that modelling activities are woven into the entire learning experience. In addition, resource limitations, such as the need for technology and materials, can be mitigated by utilising open-source software, affordable materials, or collaborating with local organisations for real-world projects. If these barriers are addressed through targeted teacher training, curriculum adaptations, and resource management, the integration of modelling activities into mathematics education can become more feasible and ultimately improve students' cognitive skills, problem-solving skills, and academic performance.

5.2.   Limitations and future suggestions

It should be noted that the research presented here is not without limitations. Firstly, this research includes only a limited number of studies on a given topic, which may lead to some bias due to the selection process. Secondly, it is difficult to fully elucidate the interactions between treatments and other factors, such as age. In addition, current research requires the use of sophisticated statistical methods to attain the desired outcomes. Furthermore, some studies may categorise modelling activities as formal curriculum components, while others may categorise them as informal, extracurricular activities. In this study, the definition of modelling activities is limited to their application in the classroom. The lack of consistency between the studies makes a direct comparison and the calculation of a pooled effect size challenging. Further research should investigate the effectiveness of modelling activities across diverse educational levels, including an investigation of the impact of modelling activities in primary, secondary, and tertiary education. This includes an investigation into the impact of modelling activities on younger students (e.g., those in elementary or middle school) compared to older students (e.g., those in high school or university). Finally, it would be beneficial to conduct long-term studies to assess the extent to which students are able to utilise the skills acquired through modelling activities in later stages of their education or careers. Specifically, future research could explore how modelling activities affect long-term learning, retention, and application of mathematical concepts. In addition, it would be useful to examine the effects of modelling activities across various age groups to gain a deeper understanding of how developmental stages may affect the outcomes. Finally, the inclusion of modelling activities in non-STEM areas could offer valuable insights into how these methods can improve critical thinking and problem-solving skills beyond mathematics and science and support a more comprehensive approach to education.

A major limitation of this study is that it relies on the PRISMA protocol and the selected databases (ProQuest, ScienceDirect, Scopus, and SpringerLink), which may not capture all relevant studies, especially those published in regional or hard-to-access journals. Although 20 studies with a total of 3047 participants were identified, the sample size and scope may not fully reflect the diversity of educational settings or student populations, limiting the applicability of the findings on a broad scale. Another important limitation is the lack of long-term data. The meta-analysis focuses primarily on short-term outcomes of modelling activities. Few studies examine how these activities affect students' sustained learning and application of mathematical concepts over time. The lack of long-term data makes it difficult to assess the lasting effects of modelling on cognitive skills, problem-solving skills, and overall achievement in mathematics. Although some studies do not fully comply with randomisation and blinding, the overall assessment of the published articles is satisfactory and indicates good quality. We recommend increasing the number of articles and extending the study period.

Author contributions

The conception or design of the work: Riyan Hidayat; The acquisition, analysis, or interpretation of data for the work: Riyan Hidayat, Ahmad Fauzi Mohd Ayub; Drafting the work or revising it critically for important intellectual content: Mohd Afifi bin Bahurudin Setambah, Nurul Hijja Mazlan.

Use of Generative-AI tools declaration

The authors declare that we did not use Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

This research received support from Fundamental Research Grant Scheme (FRGS), 2024 [grant numbers: 5540683].

Conflict of interest

The authors confirm that they have no conflicts of interest, whether financial or personal, that could have potentially influenced the findings presented in this paper.

Ethics declaration

The author declared that no ethics approval is required for the study.