A meta-analysis of STEM project-based learning on creativity

1.   Introduction

Creativity has emerged as a critical work skill in the 21st century, and the job market will increasingly demand individuals capable of utilizing and generating creativity. Creativity refers to the capacity to conceive original and beneficial concepts ^[1,2]. It not only enhances our lives but also holds a pivotal role in driving scientific innovation. Furthermore, creativity has proven its significance in addressing everyday challenges ^[3,4,5], enabling the preservation and nurturing of our well-being. As a result, creativity stands as a vital skill in our rapidly evolving world.

We find ourselves amidst the Innovation Age, where the requirements of numerous jobs have shifted and fresh employment opportunities have arisen as a result of the transformative influence of digital technologies ^[6]. The evolution of technologies has been ceaseless, and in this era dominated by platforms like Google, organizations seek individuals who possess the ability to ingeniously apply and generate knowledge. To remain competitive in this swiftly evolving landscape ^[6,7,8], students must adopt unconventional thinking and establish links between diverse ideas ^[9,10]. While many employers once expressed contentment with their employees' knowledge of subject matter, they lamented the deficiency in their creative aptitude. Furthermore, a significant number of graduates found themselves unemployed due to their creative shortcomings ^[6,11,12]. As such, it is imperative that we equip our students with the skills required to fulfill the demands of 21st-century employment opportunities.

A noteworthy correlation has been observed between science, technology, engineering, and mathematics (STEM) learning environments and students' creativity ^{[13,14,15,16]}. Researchers have found that engaging in STEM project-based learning (PBL) activities empowers students to take charge of their own learning, fostering essential 21st-century skills such as problem-solving, creativity, and communication ^[14,15,16]. Furthermore, the integration of meaningful real-world contexts and STEM content within these STEM PBL activities not only sparks students' interest in learning but also paves the way for success in STEM fields ^[13,14,17]. Given the paramount importance of ensuring equal educational opportunities for all students in the development of future STEM leaders, it becomes imperative to explore the impact of STEM learning environments on students' creativity. This exploration can lead to more widespread adoption of STEM PBL instruction, promoting the development of essential 21st-century skills and long-term success among students ^[18,19]. Therefore, by investigating this effect, valuable insights can be gained to inform instructional practices in schools that will also foster meaningful STEM learning experiences for all students.

2.   Literature review

2.1.   Science, technology, engineering, and mathematics project-based learning

Project-based learning (PBL) is an instructional approach where students collaborate to address real-world problems and apply their ideas. This method empowers learners to actively construct knowledge, reflect on their learning process, and work together with peers, while the teachers take on the role of facilitators to guide their learning journey ^[20,21]. PBL has a long history dating back to Dewey and Kilpatrick and has proven highly effective in developing essential 21st-century skills by combining practical real-world applications with rigorous academic content ^[17]. Engaging in PBL activities enhances students' understanding across various subjects and nurtures their ability to solve problems independently ^[13,18]. As a result, PBL continues to have a profound impact on education at all levels and across diverse fields.

STEM disciplines (science, technology, engineering, and mathematics) can be seamlessly integrated into PBL activities, offering students the opportunity to strengthen their STEM knowledge while collaboratively tackling relevant real-world challenges. The natural overlap between these fields makes STEM particularly well-suited for PBL, mirroring the collaborative problem-solving that occurs within STEM professions ^[13]. This interdisciplinary, student-centered STEM PBL model promotes teamwork in resolving authentic real-world problems. STEM PBL is an instructional approach characterized by its student-centered, interdisciplinary, collaborative, and occasionally technology-driven nature. This strategy heavily incorporates two learning theories: enactivism ^[22] and constructivism ^[23]. These theories underscore the significance of co-constructing knowledge and the role of the self in the learning process ^[24].

As the demand for creative and collaborative problem-solvers grows, STEM PBL has become increasingly popular as the demands for collaborative problem-solvers increase in the job market. Education plays a vital role in equipping future generations with the necessary knowledge and skills to meet the challenges of the 21st century ^[18,25]. To achieve this, STEM PBL focuses on preparing students for higher education and the modern workforce by presenting open-ended, multi-outcome challenges ^[26,27]. Research has shown that STEM PBL can improve students' understanding of mathematics and science, especially for those with lower or average achievement levels, regardless of their background ^[28,29,30]. STEM PBL aligns with educational requirements such as the common core state standards, next-generation science standards, National Council of Teachers of Mathematics principles and standards, and the Texas essential knowledge and skills standards ^[17]. A STEM PBL classroom is typically characterized by a student-centered and inquiry-based approach, where students actively solve problems and collaborate with others ^[30]. Key strategies involve fostering curiosity, posing/generating questions, making discoveries, and rigorously testing findings or intuitions to explore new solutions ^[31]. The integration of STEM PBL teaching plays a crucial role in fostering the development of engineering design, problem-solving, and higher-level thinking skills ^[30]. The primary goal of employing STEM PBL instruction is to equip students for post-secondary education and the demands of the 21st-century workforce by engaging them in solving open-ended, multi-outcome problems ^[27,30].

2.2.   Creativity

In the 21st century, creative thinking is a crucial skill, especially with the increasing use of artificial intelligence in various fields ^[4,32,33]. Creativity represents a cognitive phenomenon stemming from the utilization of commonplace mental processes, such as working memory and the proficiency to classify and manipulate objects ^[34,35]. Additionally, creativity is characterized by the ability to generate novel ideas or products that are not only relevant to the scientific context but also possess scientific usefulness or importance ^[36]. Of notable significance is the fact that the capacity for imaginative thinking can be imparted and refined—creativity does not remain an immutable innate trait ^{[37,38,39,40]}. Therefore, education should provide lessons where students can explore and grow their creativity skills.

The acknowledgment of creativity within education is on the rise. Educational institutions have begun to endorse creativity as a valuable objective within the learning process, something to be nurtured through classroom dynamics and the learning environment ^{[37,41,42,43]}. Nurturing creative thinking grants students, the freedom to explore novel concepts ^[4,37] and enriches their capacity for problem-solving by enabling them to consider diverse perspectives ^[44]. Guidelines for fostering creativity within classroom learning environments include assigning open-ended tasks, supporting risk-taking, and cultivating an atmosphere of care, acceptance, and respect ^[39,45]. Moreover, collaboratively solving STEM problems also enhances opportunities for creativity to surface. Through collaborative efforts, individuals can amalgamate their knowledge, efforts, and understanding, fostering meaningful creative processes ^[46,47]. For enduring success, educational institutions must equip students with the capacity to foster and cultivate creativity.

Unfortunately, this reality often does not well-align with the state of education. The demand for creativity has grown across many industries ^[48], and this has forced the education system to prepare students with the necessary skills and competencies such as thinking innovatively and making meaningful connections between ideas ^[7,9,10]. The current state of pedagogical approaches appears to be relatively unchanged, despite significant global shifts. Teachers often continue to rely on methods based on their own learning experiences and familiarity ^[37,49,50]. Both schools and curricula should introduce cognitive strategies with a proven ability to cultivate creative thinking skills. More attention is required in teaching cognitive strategies that have been proven to foster creative thinking skills in students ^[43,51]. Students should be proficient in using their existing knowledge to generate creative ideas and solve problems. Therefore, the present study aimed to examine the impact of STEM PBL on students' creativity. The focus for the present study was on two research questions:

(1) Are there statistically significant differences in students' creativity during STEM PBL activities?

(2) To what extent can STEM PBL affect students' creativity?

3.   Methods

3.1.   Literature search

To address the research questions, we used a meta-analytic technique. The search for relevant studies related to creativity was conducted using the following search terms: "creativity", "creative thinking", and "creative learning skill", with "STEM project-based learning (PBL)" or "STEM". A computerized literature search of the ERIC, JSTOR, CrossRef, ProQuest, and Google scholar online databases was conducted in order to identify potential studies for inclusion in the present study. The abstract and conclusion of each study was reviewed, and the studies were accepted on the basis of the following inclusion and exclusion criteria.

3.2.   Inclusion and exclusion criteria

The screening process was conducted by two researchers. Each researcher independently screened the titles and abstracts of all retrieved studies against the inclusion and exclusion criteria. To ensure consistency and reliability, the researchers underwent a calibration exercise prior to the screening. Studies that met the following criteria were included in the present meta-analysis: (1) published in English, (2) involved K–12 students, (3) measured students' creativity, (4) implemented STEM PBL in formal/informal settings, and (5) published in peer-reviewed journals/proceedings within the past 10 years (between 2014 and 2023). The searching protocol adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines ^[52]. The researchers also performed a backward search using the studies cited in other articles to locate additional sources. Initially, 149 articles were identified, and 98 articles were excluded due to duplication, irrelevant titles, or inaccessible abstracts. Upon completion of the entire search process, 51 studies that adhered to the inclusion criteria of the present meta-analysis were identified. After reviewing full-text articles according to the inclusion and exclusion criteria, eight studies with 13 effects sizes were included in the current meta-analysis. The risk of bias for each included study was assessed using Cochrane's risk of bias tool ^[52]. Figure 1 represents the flow diagram for article selection.

Figure 1.  Flow diagram for article selection.

DownLoad: Full-Size Img PowerPoint

Consequently, the meta-analysis was conducted on the basis of eight studies with 13 effect sizes (see Table 1). The program topics were diverse, covering areas such as energy, coding, diseases, and more, all under the broad umbrella of STEM PBL. The outcomes of the programs focused on creativity, including creative thinking skills, imagination, curiosity, and the creative process. The number of participants ranged from 28 to 156 at the secondary and college levels. The included studies employed experimental and quasi-experimental designs, comparing control and experimental groups or pre- and post-tests using quantitative research methods. This meta-analysis comprised a greater number of effect sizes compared with the count of included studies. This occurred due to certain studies within the compilation reporting distinct effect sizes for various creativity categories. In these instances, the effect sizes corresponding to different categories were treated as individual measurements, as each category possessed its own distinct significance under the broader realm of creativity. The included studies were all randomized controlled trials, utilizing a waitlist as the control condition. While the sample sizes of these studies exhibited variation, the cumulative total reached 702 participants. Given the limited number of studies within the meta-analysis, a moderator analysis was not performed. Nonetheless, a random effects model was employed to account for the functional differences in the characteristics of the studies. The selected articles were identified using quantitative methods. Quantitative methods, including descriptive and inferential statistics, were used in the selected studies. Most of the selected studies used instruments related to creativity or creative thinking skills developed by the authors of selected studies ^[53,54,55] except for ^[56], which adopted the instrument from ^[57]. The reliability of the instruments from the studies ranged from 0.53 to 80.

Table 1.  Included primary studies.

Authors	Program topic(s)	Program outcome	Participants	Grade level
Biazus and Mahtari (2022) [58]	Heat energy	Creative thinking skills	50	Secondary
Bicer et al. (2019) [56]	Microcontroller, solar renewable energy, 3D printing, statistics, cosmetic chemistry, cryptography, coding	Creativity within an academic context	48	Secondary
Mayasari et al. (2016) [59]	Renewable energy	Person	28	College
Mayasari et al. (2016) [59]	Renewable energy	Process	28	College
Mayasari et al. (2016) [59]	Renewable energy	Press	28	College
Sinurat et al. (2022) [53]	Free-energy aquarium	Creative thinking	156	High school
Ridlo et al. (2020) [54]	Water filtration system	Creative thinking	42	College
Parno et al. (2019) [60]	Temperature and heat	Creative thinking	34	High school
Lou et al. (2017) [61]	CaC2 steamship STEM PBL activities	Imagination	59	High school
Lou et al. (2017) [61]	CaC2 steamship STEM PBL activities	Curiosity	60	High school
Lou et al. (2017) [61]	CaC2 steamship STEM PBL activities	Challenge	60	High school
Lou et al. (2017) [61]	CaC2 steamship STEM PBL activities	Adventurousness	60	High school
Vela et al. (2019) [55]	Aerospace engineering, Coding Infectious diseases	Creativity application	49	High school

---

 | Show Table

DownLoad: CSV

3.3.   Extraction of descriptive information and inter-rater agreement

Each study was coded independently by the researchers using coding sheets to document the following information: author name, sample size, grade level, instrument, mean (M), Cohen's d, standard error (SE), and 95% confidence interval (CI). If the study did not directly report Cohen's d, the SE, or the 95% CI, the authors calculated these values using the formulas provided below ^[62]:

where ${M}_{1}, {M}_{2}$ represent the means of the groups $S{D}_{pooled} = \sqrt{\frac{\left({n}_{1}-1\right)S{D}_{1}^{2}+\left({n}_{2}-1\right)S{D}_{2}^{2}}{{n}_{1}+{n}_{2}-2}}$, in which $S{D}_{1}, S{D}_{2}$ represent the standard deviations of the groups, and ${n}_{1}, {n}_{2}$ are the sample sizes of the groups. The pooled standard deviation is

where ${{t}_{\alpha }}_{/2, df}$ is the t-value for a 95% confidence interval, and $S{E}_{d} = \sqrt{\frac{{n}_{1}+{n}_{2}}{{n}_{1}\cdot {n}_{2}}+\frac{{d}^{2}}{2({n}_{1}+{n}_{2})}}$ is the standard error of Cohen's d

After initial coding of the studies by one researcher, another researcher evaluated any points of disagreement, and studies were included in the current meta-analysis once the disagreements were resolved. The inter-rater agreement related to determining the continuous variables was calculated by House et al.'s formula (1981) ^[62]:

The result of the calculation was 100%, which showed excellent inter-rater agreement.

3.4.   Analysis

A random effects model was employed for the meta-analysis of this study due to the functional variations in the characteristics across the included studies. These variations encompassed diverse settings, types of STEM PBL treatments, grade ranges, and more. Consequently, it was expected that the effect sizes would differ among the individual studies within the meta-analysis. The random-effects model accounts for these variations, allowing for the generalization of a common effect size. This approach strikes an appropriate balance between acknowledging differences and estimating the pooled effect. For the analysis, Cohen's d effect size was computed for each study. To mitigate sampling bias, the effect sizes were weighted using the sample sizes through a weight function. By employing the weighted effect sizes, the researchers determined an overall mean effect size and standard errors for the meta-analysis. The comprehensive Meta-Analysis V4 package was utilized to compute the effect sizes across the studies and generate forest plots. To assess heterogeneity, Cochran's Q was calculated by summing the squared deviations between the effects of individual studies and the pooled effect across all studies. Cochran's Q helped determine whether the variations between studies were greater than what would be expected due to chance alone. The contribution of each study was weighted in the same manner as in the meta-analysis. Below is the formula for Cochran's Q ^[62]:

$\widehat{{\theta }_{i}}$: Effect estimator of the $i$th study;

${\widehat{\theta }}_{w} = \frac{{\sum }_{i}\widehat{{w}_{i}}\widehat{{\theta }_{i}}}{{\sum }_{i}\widehat{{w}_{i}}}$ : Weighted average of the estimators of the effects;

$\widehat{{w}_{i}}$ : Inverse of the variance estimator of the $i$th effect estimator;

In addition, ${I}^{2}$, which describes the percentage of total variation across studies, was proposed as a method to quantify heterogeneity ^[63]. Below is the formula for Cochran's ${I}^{2}$:

where $Q$ is Cochran's Q and $df$ is degree of freedom.

4.   Results

The overall effect size was 3.888 with a 95% confidence interval of 3.609 to 4.166. The overall effect size in the universe of comparable studies could fall anywhere in this interval. Given that the focus of the current study revolves around assessing the extent to which STEM PBL contributes to the enhancement of students' creativity, a positive mean effect size would signify the efficacy of STEM PBL in fostering creativity. Consequently, the calculated weighted mean effect size presented in this analysis might straightforwardly indicate the favorable impact of interventions involving STEM PBL on the augmentation of students' creative abilities. The Z-value tests the null hypothesis that the mean effect size is zero. The Z-value is 27.366 with p < 0.001. Using a criterion alpha of 0.050, we reject the null hypothesis and conclude that in the universe of populations similar to those in the analysis, the mean effect size is not precisely zero. Figure 1 represents a forest plot, as well as Cohen's ds, the SEs, and the 95% CIs of d.

Table 2.  Cohen's d, SE, and CIs of the impact of STEM PBL on creativity.

Authors	Cohen's d	SE	Approximate 95% CI of d		Forest plot
			Minimum	Maximum
Biazus and Mahtari (2022) [58]	4.11	0.35	3.41	4.80
Bicer et al. (2019) [56]	3.77	0.41	2.97	4.57
Mayasari et al. (2016) [59]	2.25	0.48	1.30	3.19
Mayasari et al. (2016) [59]	3.79	0.63	2.56	5.03
Mayasari et al. (2016) [59]	2.65	0.52	1.63	3.67
Sinurat et al. (2022) [53]	4.43	0.67	3.12	5.73
Ridlo et al. (2020) [54]	3.74	0.36	3.03	4.45
Parno et al. (2019) [60]	5.52	0.53	4.48	6.56
Lou et al. (2017) [61]	3.71	0.18	3.35	4.07
Lou et al. (2017) [61]	3.85	0.18	3.48	4.20
Lou et al. (2017) [61]	3.99	0.18	3.63	4.36
Lou et al. (2017) [61]	4.14	0.18	3.78	4.50
Vela et al. (2019) [55]	4.28	0.20	3.89	4.68
Note: In the forest plot, the horizontal line represents the 95% confidence intervals of each study. The short vertical line represents the effect size of the study. The yellow highlighted line on the bottom represents the overall effects across 13 effect sizes.

---

 | Show Table

DownLoad: CSV

The Q-statistic provides a test of the null hypothesis that all studies in the analysis share a common effect size. The Q-value is 34.691 with 12 degrees of freedom and p = 0.001. Using an alpha criterion of 0.100, we can reject the null hypothesis that the true effect size is the same in all these studies. The I² statistic is 65%, which tells us that some 65% of the variance in the observed effects reflects variance in the true effects rather than sampling error. Figure 2 shows a funnel plot of the SEs and effect sizes of the selected studies. The funnel plot revealed that there was no publication bias in the present study.

Figure 2.  Funnel plot of included studies.

DownLoad: Full-Size Img PowerPoint

5.   Discussion and conclusions

The aim of this meta-analysis was to address the following research questions: Does STEM PBL have a significant impact on students' creativity, and to what extent? The findings from this study revealed statistically significant effects of STEM PBL on students' creativity, with an overall effect size of 0.388 (95% CI = [3.609, 4.166]). These results underscore the vital role of STEM PBL in facilitating students' learning, particularly in nurturing their creativity. Additionally, the findings can help further contextualize its practical significance. In the realm of education, creativity has emerged as a crucial factor. For example, in the United States, the National Council of Teachers of Mathematics has emphasized the significance of fostering students' creativity, aligning with the demands of 21st-century skills and closely relating to the competencies advocated by the Common Core State Standards ^[64]. In Korea, the Ministry of Education ^[65] has underscored the significance of nurturing students' creativity. For instance, within the revised 2022 curriculum, they emphasized that fostering creative thinking and creative problem-solving skills is a pivotal educational objective, aimed at cultivating students, transforming them into individuals capable of innovative thinking. For example, they said, "It nurtures the ability to creatively solve problems by integrating knowledge and experiences from diverse fields, and to proactively respond to new situations." (p.7, ^[65]) This emphasis on cross-disciplinary integration aligns well with the defining features of STEM PBL.

By fostering interdisciplinary learning and cultivating active engagement through STEM PBL, students have the opportunity to mature into autonomous learners. This represents a fundamental contrast to traditional instruction methods. Approaches like lectures, teacher-centered teaching, and an emphasis on rote memorization hinge on the direct dissemination of mathematical information ^[66]. In contrast, STEM PBL, acknowledged as a contemporary and innovative pedagogical approach, empowers students to take ownership of their learning as they immerse themselves in meaningful activities. Within this framework, the conveyance of mathematical information takes a different form–students independently process and internalize the information, thus promoting a deeper understanding ^[67]. The cultivation of self-directed learning in autonomous students contributes to their development as creative thinkers.

The socio-cultural environment and cognitive traits inherent in STEM PBL contribute to the emergence of creativity. Socio-cultural factors have consistently been associated with heightened creativity across extensive time periods ^[68], particularly in scenarios where students are actively participating in STEM PBL endeavors ^[27]. The cognitive aspect of creativity centers around comprehending the mental frameworks and mechanisms that underlie human thinking ^[68]. Creativity involves the application of cognitive processes to manipulate the knowledge already retained in an individual's memory, as outlined in ^[69].

In spite of the positive impact of STEM PBL on creativity, as seen in the results of the current study, there have been several concerns. Creativity is among the 21st-century skills that are needed by students in facing the advance of technology and preparing for their future career. According to teacher interviews, there are still many teachers who measure cognitive aspects. In this case, there is an indication that students have a lack of skills, especially in creativity. Teachers have not trained students to strengthen their creativity, even though the curriculum that has been developed puts more emphasis on the creativity aspect. Creativity is one of important skills that should be fostered by students ^[70]. Creativity refers to the creation of a novel and appropriate response, product, or solution to an open-ended task ^[71]. If creativity relates to learning and technology, it will produce a high quality of work. A recent study showed that technology allows the students to construct several media that can help them to produce high-quality work in the creativity context ^[72]. STEM project-based learning has a chance to have a positive impact on creativity because students will develop their own idea to create the product.

Despite the positive impact of STEM PBL on creativity, it is worth noting that there remains a limited number of studies available on this topic. Although the recognition of creativity within the realm of education is on an upward trend, not enough research has been conducted in education to embrace creativity as a valuable learning goal. As the demand for creativity in the job market is growing ^[50], students should be prepared to think innovatively and learn to establish meaningful connections between the concepts they learn in their classroom. There is a pressing need to research how to teach STEM-related subjects that have demonstrated efficacy in fostering creative thinking skills among students.

In conclusion, the STEM PBL approach affects students' creative thinking skills. Schools and curricula need to use this research-proven cognitive strategy that can enhance students' creative thinking skills. For sustainable success, educational institutions must equip students with the capacity to nurture and develop creativity. Exposing students to STEM PBL will allow them to become adept at leveraging their existing knowledge to generate creative ideas and solve problems.

The constraints of this meta-analysis emphasize the need for a cautious interpretation of the results. The analysis encompasses the timeframe from 2014 to 2023, meaning that literature published prior to 2014 and beyond 2023 has not been incorporated into this review. Additionally, while utilizing various search engines to gather literature, the extent of gray literature included was restricted. For future studies, moderator analysis could be employed to explore the influence of specific variables on the outcomes, provided that a larger and more diverse set of studies becomes available Moreover, the array of instruments employed in these investigations, frequently crafted by the researchers themselves, underscores the deficiency in established theoretical frameworks and appraisals within the realm of STEM PBL and creativity. The majority of these instruments consist of surveys or tests administered to students. This implies that the creativity evaluated in these studies primarily hinges on self-reported accounts, potentially resulting in the participants possessing an imprecise grasp of their own creative prowess. Some instruments have exhibited relatively low reliability, which could affect the results of the included studies and underscores the need for cautious interpretation. Additionally, the learning environment might have influenced the extent to which their creativity was actualized, thereby constraining the robustness of the measurement. The results of this study provide actionable insights for educators and stakeholders, encouraging the implementation of STEM PBL in learning processes and approaches to better nurture creativity in students. Across the included studies, a variety of topics for STEM PBL programs were developed and implemented. Real-world situations serve as a rich source for generating STEM PBL activities, which, in turn, can effectively foster students' creativity. Recognizing the significant positive impact of STEM PBL on fostering creativity, it is essential to invest efforts in developing a diverse and comprehensive range of STEM PBL curricula. Such curricula should be designed to address various educational contexts and real-world challenges, providing meaningful engagement for students while fostering their creative potential.

Use of Generative AI tools declaration

The authors declare they have not used artificial intelligence (AI) tools in the creation of this article.

Acknowledgments

We would like to thank the constructive feedback provided by the reviewers.

Conflict of interest

The authors declare that there are no conflicts of interest in this paper.

Ethics declaration

The authors declare that the ethics committee approval was waived for the study.