The Impact of Refutation Texts in Scientific Conception: Meta-analysis

Mohd Zaidi Bin Amiruddin, Achmad Samsudin, Andi Suhandi, Ida Kaniawati, Bayram COŞTU

Abstract


This research aimed to analyze the role and impact of refutational text on student conceptions and learning outcomes. Utilizing meta-analysis methods, this study sourced articles from the Scopus and Web of Science databases, covering publications from 2001 to 2023. The article selection process adhered to the PRISMA technique, implemented systematically in several stages to ensure rigor and transparency. The meta-analysis used the random-effects model to account for variability among studies, offering a more generalized understanding of the effects of refutational text across different contexts and populations. The results were synthesized through a Forest plot, revealing a standard mean difference with a P value of <0.01, indicating a statistically significant effect of refutational text on students’ conceptions. This suggests that the use of refutational text substantially influences learning, particularly in addressing misconceptions. Additionally, the study highlighted that while the potential for applying refutational texts in Indonesia is promising, the number of global and internationally indexed publications on this topic remains limited. This lack of extensive research limits the broader application of refutational text in educational contexts. However, the findings suggest that refutational text holds significant promise for science education, especially for topics where students commonly hold misconceptions about difficult concepts. Further research is recommended to explore its full potential and to encourage the publication and wider application of refutation texts in science education.      

 

Keywords: refutation text, meta-analysis, conception, conceptual change.

Full Text:

PDF

References


Adesope, O. O., Cavagnetto, A., Hunsu, N. J., Anguiano, C., & Lloyd, J. (2017). Comparative effects of computer-based concept maps, refutational texts, and expository texts on science learning. Journal of Educational Computing Research, 55(1), 46–69.

Adnan, Z. (2009). Some potential problems for research articles written by Indonesian academics when submitted to international English language journals. The Asian EFL Journal Quarterly, 11(1), 107-125.

Afonso, J., Ramirez-Campillo, R., Clemente, F. M., Büttner, F. C., & Andrade, R. (2024). The perils of misinterpreting and misusing “publication Bias” in meta-analyses: an education review on funnel plot-based methods. Sports medicine, 54(2), 257-269.

Aguilar, S. J., Polikoff, M. S., & Sinatra, G. M. (2019). Refutation Texts: A New Approach to Changing Public Misconceptions About Education Policy. July. https://doi.org/10.3102/0013189X19849416

Azizah, U. A., & Budiman, A. (2017). Challenges in writing academic papers for international publication among Indonesian graduates students. JEELS (Journal of English Education and Linguistics Studies), 4(2), 175-197.

Azer, S. A., Dupras, D. M., & Azer, S. (2014). Writing for publication in medical education in high impact journals. European Review for Medical & Pharmacological Sciences, 18(19).

Baas, J., Schotten, M., Plume, A., Côté, G., & Karimi, R. (2020). Scopus as a curated, high-quality bibliometric data source for academic research in quantitative science studies. Quantitative science studies, 1(1), 377-386.

Basyit, A., Syuhendri, S., & Fathurohman, A. (2024). Analisis kebutuhan e-modul berbasis perubahan konseptual pada materi fluida dinamis Kelas XI SMA. JIIP-Jurnal Ilmiah Ilmu Pendidikan, 7(7), 7110-7117.

Beker, K., Kim, J., Boekel, M. Van, van den Broek, P. W., & Kendeou, P. (2019). Refutation texts enhance spontaneous transfer of knowledge. Contemporary Educational Psychology, null, null. https://doi.org/10.1016/J.CEDPSYCH.2018.11.004

Braasch, J. L. G., Goldman, S., & Wiley, J. (2013). The influences of text and reader characteristics on learning from refutations in science texts. Journal of Educational Psychology, 105, 561–578. https://doi.org/10.1037/A0032627

Broughton, S. H., Sinatra, G., & Reynolds, R. (2010). The nature of the refutation text effect: an investigation of attention allocation. The Journal of Educational Research, 103, 407–423. https://doi.org/10.1080/00220670903383101

Caleon, I., & Subramaniam, R. (2013). Addressing students’ alternative conceptions on the propagation of periodic waves using a refutational text. Physics Education, 48(5), 657.

Calik, M., Kolomuç, A., & Karagölge, Z. (2010). The effect of conceptual change pedagogy on students’ conceptions of rate of reaction. Journal of Science Education and Technology, 19, 422–433.

Cochrane, K. L., Butterworth, D. S., Hilborn, R., Parma, A. M., Plagányi, É. E., & Sissenwine, M. P. (2024). Errors and bias in marine conservation and fisheries literature: Their impact on policies and perceptions. Marine Policy, 168, 106329.

Cohen, L., Manion, L., & Morrison, K. (2018). Research Methods in Education (8th ed.).

Routledge Taylor & Francis Group. https://doi.org/10.4324/9781315456539

Cheema, H. A., Shahid, A., Ehsan, M., & Ayyan, M. (2022). The misuse of funnel plots in meta-analyses of proportions: are they really useful?. Clinical Kidney Journal, 15(6), 1209-1210.

Danielson, R. W., Jacobson, N. G., Patall, E. A., Sinatra, G. M., Adesope, O. O., Kennedy, A. A., ... & Sunday, O. J. (2025). The effectiveness of refutation text in confronting scientific misconceptions: A meta-analysis. Educational Psychologist, 60(1), 23-47.

Dersch, A., Renkl, A., & Eitel, A. (2022). Personalized refutation texts best stimulate teachers’ conceptual change about multimedia learning. Journal of Computer Assisted Learning, 38(4), 977–992.

Devianty, D., & Syuhendri, S. (2023). Preliminary development of refutation texts for conceptual change on lunar phases. 2nd International Conference on Science Education and Sciences 2022 (ICSES 2022), 212–222.

Diakidoy, I. A. N., Mouskounti, T., & Ioannides, C. (2011). Comprehension and learning from refutation and expository texts. Reading Research Quarterly, 46(1), 22–38. https://doi.org/https://doi.org/10.1598/RRQ.46.1.2

Djudin, T. (2021). Promoting students’ conceptual change by integrating the 3-2-1 reading technique with refutation text in the physics learning of buoyancy. Journal of Turkish Science Education, 18(2), 290–303.

Dole, J. A. (2000). Readers, texts and conceptual change learning. Reading & Writing Quarterly, 16(2), 99–118.

Dwivedi, Y. K., Jeyaraj, A., Hughes, L., Davies, G. H., Ahuja, M., Albashrawi, M. A., ... & Walton, P. (2024). “Real impact”: Challenges and opportunities in bridging the gap between research and practice–Making a difference in industry, policy, and society. International Journal of Information Management, 102750.

Fernandes, A., Shukla, B., & Fardoun, H. (2022). Fostering reputation of higher education institutions in international ranking by means of diversity of international collaboration. Information Sciences Letters, 11(4), 1137-1143.

Fratiwi, N. J., Samsudin, A., Ramalis, T. R., Saregar, A., Diani, R., & Ravanis, K. (2020). Developing MeMoRI on newton’s laws: for identifying students’ mental models. European Journal of Educational Research, 9(2), 699–708.

Fritz, C. O., Morris, P. E., & Richler, J. J. (2012). Effect size estimates: current use, calculations, and interpretation. Journal of Experimental Psychology: General, 141(1), 2. https://doi.org/https://doi.org/10.1037/a0024338

Guzzetti, B. J., Williams, W. O., Skeels, S. A., & Wu, S. M. (1997). Influence of text structure on learning counterintuitive physics concepts. Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching, 34(7), 701–719.

Halverson, L. R., Graham, C. R., Spring, K. J., & Drysdale, J. S. (2012). An analysis of high impact scholarship and publication trends in blended learning. Distance Education, 33(3), 381-413.

Harsch, R. M., & Kendeou, P. (2023). Learning from refutation texts about scientific topics with analogical and causal explanations. Contemporary Educational Psychology, 73(March), 102172. https://doi.org/10.1016/j.cedpsych.2023.102172

Heddy, B. C., & Sinatra, G. M. (2013). Transforming misconceptions: Using transformative experience to promote positive affect and conceptual change in students learning about biological evolution. Science Education, 97(5), 723–744.

Hendratmoko, A. F., Madlazim, M., Widodo, W., Suyono, S., & Supardi, Z. A. I. (2024). Inquiry and debate in science learning: potential strategy for improving students' scientific argumentation skills. International Journal of Education in Mathematics, Science and Technology, 12(1), 114-138.

Jacobson, N. G., Thacker, I., & Sinatra, G. (2021). Here’s hoping it’s not just text structure: the role of emotions in knowledge revision and the backfire effect. Discourse Processes, 59, 13–35. https://doi.org/10.1080/0163853X.2021.1925059

Jacob, M., & Meek, V. L. (2013). Scientific mobility and international research networks: trends and policy tools for promoting research excellence and capacity building. Studies in higher education, 38(3), 331-344.

Jin, G., Ramazan, O., & Danielson, R. W. (2024). Effects of refutational texts and seductive pictures on conceptual change. The Journal of Experimental Education, 92(4), 605-625.

Kendeou, P., Walsh, E. K., Smith, E. R., & O’Brien, E. J. (2014). Knowledge revision processes in refutation texts. Discourse Processes, 51, 374–397. https://doi.org/10.1080/0163853X.2014.913961

Khan, T., Emon, M. M. H., Rahman, M. A., Hamid, A. B. A., & Yaakub, N. I. (2025). Bridging the gap: Realizing GreenTech potential. In AI and green technology applications in society (pp. 91-122). IGI Global Scientific Publishing.

Kwiek, M. (2018). International research collaboration and international research orientation: Comparative findings about European academics. Journal of Studies in International Education, 22(2), 136-160.

Lai, E. R. (2011). Critical thinking: A literature review. Pearson's Research Reports, 6(1), 40-41.

Manik, L. P., Akbar, Z., Yaman, A., & Indrawati, A. (2022). Indonesian scientists’ behavior relative to research data governance in preventing WMD-applicable technology transfer. Publications, 10(4), 50.

Mariati, M., Junaidi, A., & Napitupulu, E. (2023). Interaksi Refutation Text Dan Explanation Text Terhadap Miskonsepsi Mahasiswa. Liabilities (Jurnal Pendidikan Akuntansi), 6(3), 1-9.

Mason, L., Baldi, R., Ronco, S. Di, Scrimin, S., Danielson, R., & Sinatra, G. (2017). Textual and graphical refutations: Effects on conceptual change learning. Contemporary Educational Psychology, 49, 275–288. https://doi.org/10.1016/J.CEDPSYCH.2017.03.007

Mason, L., Zaccoletti, S., Carretti, B., Scrimin, S., & Diakidoy, I.-A. N. (2019). The Role of Inhibition in Conceptual Learning from Refutation and Standard Expository Texts. International Journal of Science and Mathematics Education, 17, 483–501. https://doi.org/10.1007/S10763-017-9874-7

Mufida, S. N., Samsudin, A., Suhendi, E., Kaniawati, I., & Novia, H. (2023). CoSiReT: innovation of ret (refutation texts) to reduce students’ misconceptions concerning transverse waves. Jurnal Penelitian Pendidikan IPA, 9(11), 9363–9371.

Muliyani, R. (2018). The Refutation Text In Poe To Reduce The Quantity Of Students’misconception. Journal of Education Research and Evaluation, 2(2), 62–71.

Nadiyah, A., Ilhamsyah, F., Alrefi, A., Al-Abyadh, M. H. A., & Roman, N. (2023). Analysing The Road Of Self-Efficacy Research In Indonesia. Pamomong: Journal of Islamic Educational Counseling, 4(1), 1-13.

PinarbaŞi, T., Canpolat, N., BayrakÇeken, S., & Geban, Ö. (2006). An investigation of effectiveness of conceptual change text-oriented instruction on students’ understanding of solution concepts. Research in Science Education, 36, 313–335.

Poehnl, S., & Bogner, F. X. (2013). A modified refutation text design: Effects on instructional efficiency for experts and novices. Educational Research and Evaluation, 19(5), 402–425. https://doi.org/https://doi.org/10.1080/13803611.2013.789229

Prinz, A., Golke, S., & Wittwer, J. (2019). Refutation texts compensate for detrimental effects of misconceptions on comprehension and metacomprehension accuracy and support transfer. Journal of Educational Psychology, 111(6), 957.

Prinz, A., Golke, S., & Wittwer, J. (2021). Counteracting detrimental effects of misconceptions on learning and metacomprehension accuracy: The utility of refutation texts and think sheets. Instructional Science, 49, 165–195. https://doi.org/10.1007/s11251-021-09535-8

Rafi, M., JianMing, Z., & Ahmad, K. (2019). Evaluating the impact of digital library database resources on the productivity of academic research. Information discovery and delivery, 47(1), 42-52.

Prinz, A., Kollmer, J., Flick, L., Renkl, A., & Eitel, A. (2022). Refuting student teachers’ misconceptions about multimedia learning. Instructional Science, 50(1), 89–110.

Rodgers, M. A., & Pustejovsky, J. E. (2021). Evaluating meta-analytic methods to detect selective reporting in the presence of dependent effect sizes. Psychological methods, 26(2), 141.

Salager-Meyer, F. (2008). Scientific publishing in developing countries: Challenges for the future. Journal of English for academic purposes, 7(2), 121-132.

Samsudin, A., Rusdiana, D., Efendi, R., Fratiwi, N. J., Aminudin, A. H., & Adimayuda, R. (2021). Development of Predict-Observe-Explain (POE) strategy assisted by rebuttal texts on newton’s law material with rasch analysis. Tadris: Jurnal Keguruan Dan Ilmu Tarbiyah, 6(1).

Samsudin, A., Suhandi, A., Linuwih, S., Kaniawati, I., Fauzi, D. M., Aminudin, A. H., Fratiwi, N. J., & Adimayuda, R. (2023). Preliminary development of simulation on refutational-text (sort) to change conception of physics: Rasch and nvivo analysis. AIP Conference Proceedings, 2614(1).

Schroeder, N. L., & Kucera, A. (2022). Refutation text facilitates learning: a meta-analysis of between-subjects experiments. Educational Psychology Review, 34, 957–987. https://doi.org/10.1007/s10648-021-09656-z

Scott, P., Asoko, H., & Leach, J. (2013). Student conceptions and conceptual learning in science. In Handbook of research on science education (pp. 31–56). Routledge.

Sinatra, G. M., & Broughton, S. H. (2011). Bridging reading comprehension and conceptual change in science education: The promise of refutation text. Reading Research Quarterly, 46(4), 374–393.

Suriano, R., Plebe, A., Acciai, A., & Fabio, R. A. (2025). Student interaction with ChatGPT can promote complex critical thinking skills. Learning and Instruction, 95, 102011.

Taylor, A. K., & Kowalski, P. (2023). Refuting misconceptions: One size does not fit all. Scholarship of Teaching and Learning in Psychology. https://doi.org/https://doi.org/10.1037/stl0000384

Thacker, I., Sinatra, G. M., Muis, K. R., Danielson, R. W., Pekrun, R., Winne, P. H., & Chevrier, M. (2020). Using persuasive refutation texts to prompt attitudinal and conceptual change. Journal of Educational Psychology, 112(6), 1085.

Tippett, C. D. (2010). Refutation text in science education: A review of two decades of research. International Journal of Science and Mathematics Education, 8, 951–970.

Vaughn, A. R. (2018). Effectiveness of a Refutation Text with Appeals to Expertise in Establishing Credibility for Conceptual Change: A Mixed Methods Study. University of Cincinnati.

Wahyuningsih, I. T., Djudin, T., & Oktavianty, E. (2019). Pengembangan refutation text untuk meremediasi miskonsepsi peserta didik pada materi fluida dinamis. Jurnal Pendidikan dan Pembelajaran Khatulistiwa (JPPK), 8(10).

Will, K. K., Masad, A., Vlach, H. A., & Kendeou, P. (2019). The effects of refutation texts on generating explanations. Learning and Individual Differences, null, null. https://doi.org/10.1016/J.LINDIF.2018.12.002

Zengilowski, A., Schuetze, B. A., Nash, B. L., & Schallert, D. L. (2021). A critical review of the refutation text literature: Methodological confounds, theoretical problems, and possible solutions. Educational Psychologist, 56(3), 175–195. https://doi.org/https://doi.org/10.1080/00461520.2020.1861948

Zengilowski, A., Schuetze, B. A., Nash, B. L., & Schallert, D. L. (2021). A critical review of the refutation text literature: Methodological confounds, theoretical problems, and possible solutions. Educational Psychologist, 56(3), 175–195.

Zwetsloot, P., Naald, M. Van Der, Sena, E. S., Howells, D. W., Inthout, J., Groot, J. A. H. De, Chamuleau, S. A. J., Macleod, M. R., & Wever, K. E. (2017). Standardized mean differences cause funnel plot distortion in publication bias assessments. 1–20.




DOI: https://doi.org/10.23960/jpmipa.v26i1.pp153-168

Refbacks

  • There are currently no refbacks.


Copyright (c) 2025 Jurnal Pendidikan MIPA

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Creative Commons License
The copyright is reserved to The Jurnal Pendidikan Progresif that is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.