Mapping Misconceptions in Heat and Temperature: Rasch Analysis of Four-Tier Diagnostic Responses
Understanding of heat and temperature has been a major obstacle in students' learning of thermodynamics, but many diagnostic studies still report descriptive data on the frequencies of students' responses, which lack evidence about the psychometric quality of the measurement, the ranking of items, and the validity of the response patterns. This paper aims to address this by combining a four-tier diagnostic test with the Rasch Partial Credit Model (PCM) to assess Grade XII students' conceptual knowledge of heat and temperature and to evaluate the diagnostic test's psychometric quality. We surveyed 60 senior high school students in Bandung, Indonesia, using a quantitative approach. They responded to a diagnostic test with four-tiered items on the concepts of temperature, thermal expansion, heating effect on temperature, states of matter, and heat transfer. We classified these responses into conceptual categories and applied Rasch-PCM to assess reliability, item and person fit, unidimensionality, category functioning, gender-based differential item functioning (DIF), Wright map, and the match between difficulty and ability. The findings show the instrument had satisfactory psychometric characteristics, including a reliable hierarchy of item difficulty, acceptable fit to the model, essential unidimensionality, and mostly well-functioning response categories. There was little evidence of measurement bias as items functioned similarly for both males and females. The diagnostic map indicated that students' concepts were characterized by partial understanding and misunderstanding, and by ongoing confusion among heat, temperature, and thermal energy. The most challenging concepts were density change upon heating, convection, thermal expansion on the particle level, thermal feelings, and mechanisms of heat transfer. The person-fit and scalogram analyses also showed irregular response patterns of a minority group of students, suggesting a potential for guessing, disjointed reasoning, or unstable understanding. These findings show that the use of four-tier diagnostic responses with Rasch-PCM offers a higher-standard approach to identifying misconception structures than descriptive diagnostic analysis.
Keywords: student misconceptions, four-tier diagnostic, rasch analysis, heat, temperature.
Agustina, I., Astuti, D., Bhakti, Y. B., Prasetya, R., & Rahmawati, Y. (2023). Four tier-relativity diagnostic test (4T-RDT) to Identify Student Misconception. JIPF (Jurnal Ilmu Pendidikan Fisika), 8(1), 75–84. https://doi.org/10.26737/JIPF.V8I1.3668
Ali, K., Slade, A., Kay, E. J., Zahra, D., Chatterjee, A., & Tredwin, C. (2017). Application of Rasch analysis in the development and psychometric evaluation of the dental undergraduates' preparedness assessment scale. European Journal of Dental Education, 21(4), e135–e141. https://doi.org/10.1111/eje.12236
Avinç, E., & Doğan, F. (2024). Digital literacy scale: Validity and reliability study with the rasch model. Education and Information Technologies 2024 29:17, 29(17), 22895–22941. https://doi.org/10.1007/S10639-024-12662-7
Bandhu, D., Khadir, M., Kaushik, A., Sharma, S., Ali, H. A., & Jain, A. (2023). Innovative approaches to thermal management in next-generation electronics. E3s Web of Conferences, 430. https://doi.org/10.1051/e3sconf/202343001139
Bezlutsky, V., Melnyk, O., Emelyanova, T., Lisnychiy, V., Lytvynenko, A., & Sova, S. (2024). Temperature loads in railway structures and general engineering practice. Transport Means Proceedings of the International Conference, 2024-Octob, 581–586. https://doi.org/10.5755/e01.2351-7034.2024.P581-586
Boone, W. J., Yale, M. S., & Staver, J. R. (2014). Rasch analysis in the human sciences. In Rasch Analysis in the Human Sciences. Springer. https://doi.org/10.1007/978-94-007-6857-4
Busyairi, A., Munandar, R., Apsari, P. A. D., Wahyuni, A., Nurhasanah, Arni, K. J., Walihah, Z., & Diarta, M. H. (2022). Identification of prospective physics teacher’s misconceptions of temperature and heat concept using the three tier test. Amplitudo Journal of Science and Technology Innovation, 1(2), 48–53. https://doi.org/10.56566/amplitudo.v1i2.9
Caleon, I. S., & Subramaniam, R. (2010). Do students know What they know and what they don’t know? Using a four-tier diagnostic test to assess the nature of students’ alternative conceptions. Research in Science Education, 40(3), 313–337. https://doi.org/10.1007/s11165-009-9122-4
Çelikkanlı, N. Ö., & Kızılcık, H. Ş. (2022). A review of studies about four-tier diagnostic tests in physics education. Journal of Turkish Science Education, 19(4), 1291–1311. https://doi.org/10.36681/tused.2022.175
Dhyaaldian, S. M. A., Kadhim, Q. K., Mutlak, D. A., Neamah, N. R., Kareem, Z. H., Hamad, D. A., Tuama, J. H., & Qasim, M. S. (2022). A comparison of polytomous rasch models for the analysis of c-tests. International Journal of Language Testing, 12(2), 107–117. https://doi.org/10.22034/IJLT.2022.157128
Diani, R., Alfin, J., Anggraeni, Y. M., Mustari, M., & Fujiani, D. (2019). Four-Tier diagnostic test with certainty of response index on the concepts of fluid. Journal of Physics Conference Series, 1155(1). https://doi.org/10.1088/1742-6596/1155/1/012078
Farida, F., Alamsyah, Y. A., Anggoro, B. S., Andari, T., & Lusiana, R. (2024). Rasch measurement validation of an assessment tool for measuring students’ creative problem-solving through the use of ict | validación de una herramienta de evaluación basada en el modelo de rasch para medir la resolución creativa de problemas en estudiantes. Pixel Bit Revista De Medios Y Educación, 71, 83–106. https://doi.org/10.12795/pixelbit.107973
Fauzan, A., Ega, H. M., Sigalingging, J. A., & Nugroho, Y. S. (2021). Analysis of heat gains from flat plate heater measured using multi-axis heat flux sensors. Evergreen, 8(4), 844–849. https://doi.org/10.5109/4742130
Fenditasari, K., Jumadi, Istiyono, E., & Hendra. (2020). Identification of misconceptions on heat and temperature among physics education students using four-tier diagnostic test. Journal of Physics Conference Series, 1470(1). https://doi.org/10.1088/1742-6596/1470/1/012055
Hawkins, R. J., Hawkins, J., Tremblay, B., Wiles, L., & Higgins, K. (2024). Use of the rasch model for fit statistics and rating scale diagnosis for the student perception appraisal. Journal of Nursing Measurement, 32(3), 391–403. https://doi.org/10.1891/JNM-2022-0122
Hermita, N., Sakinah, S., Wijaya, T. T., Vebrianto, R., Alim, J. A., Putra, Z. H., Fauza, N., Dipuja, D. A., Pereira, J., & Jihe, C. (2021). Item analysis of heat transfer concept using rasch model in elementary school. Journal of Physics Conference Series, 2049(1). https://doi.org/10.1088/1742-6596/2049/1/012058
Hermita, N., Suhandi, A., Syaodih, E., Samsudin, A., Isjoni, & Rosa, F. (2017). Assessing pre-service elementary school teachers’ alternative conceptions through a four-tier diagnostic test on magnetism concepts. Advanced Science Letters, 23(11), 10910–10912. https://doi.org/10.1166/asl.2017.10184
Hikmah, F. N., Sukarelawan, Moh. I., Nurjannah, T., & Djumati, J. (2021). Elaboration of high school student’s metacognition awareness on heat and temperature material: wright map in rasch model. Indonesian Journal of Science and Mathematics Education, 4(2), 172–182. https://doi.org/10.24042/IJSME.V4I2.9488
Ismail, M. S., Din, M. S. H., & Jusoh, M. S. (2021). Predictive modelling using Rasch’s person-item map: Intrepreting and assessing in Malaysia manufacturing firms. AIP Conference Proceedings, 2347. https://doi.org/10.1063/5.0052973
Istiyono, E. (2022). Diagnostic tests as an important pillar in today’s physics learning: four-tier diagnostic test a comprehensive diagnostic test solution. Journal of Physics Conference Series, 2392(1). https://doi.org/10.1088/1742-6596/2392/1/012001
Istiyono, E., Fenditasari, K., Ayub, M. R. S., Saepuzaman, D., & Dwandaru, W. S. B. (2024). An eight-category partial credit model as very appropriate for four-tier diagnostic test scoring in physics learning. AIP Conference Proceedings, 2622(1). https://doi.org/10.1063/5.0133862
Istiyono, E., Shanti, M. R. S., Saepuzaman, D., Dwandaru, W. S. B., & Zakwandi, R. (2022). A four tier web based assessment with eight categories diagnostic. Proceedings International Conference on Education and Technology ICET, 2022-Octob, 190–196. https://doi.org/10.1109/ICET56879.2022.9990847
Istiyono, E., Sunu Brams Dwandaru, W., Fenditasari, K., Ayub, M. R. S. S. N., & Saepuzaman, D. (2023). The development of a four-tier diagnostic test based on modern test theory in physics education. European Journal of Educational Research, 12(1), 371–385. https://doi.org/10.12973/eu-jer.12.1.371
Jumadi, J., Sukarelawan, M. I., & Kuswanto, H. (2023). An investigation of item bias in the four-tier diagnostic test using Rasch model. International Journal of Evaluation and Research in Education, 12(2), 622–629. https://doi.org/10.11591/ijere.v12i2.22845
Kaltakci-Gurel, D., Eryilmaz, A., & McDermott, L. C. (2017). Development and application of a four-tier test to assess pre-service physics teachers’ misconceptions about geometrical optics. Research in Science and Technological Education, 35(2), 238–260. https://doi.org/10.1080/02635143.2017.1310094
Li, X., Su, Z., Wang, L., Li, J., & Diao, Y. (2024). Psychometric evaluation of the pictorial scale of perceived movement skill competence in Chinese children: An item response theory analysis. Child Care Health and Development, 50(4). https://doi.org/10.1111/cch.13275
Munggarani, M. E., Supriyati, Y., & Astra, I. M. (2021). Identifying high school students’ misconceptions using digital four-tier diagnostic tests in distance learning. Journal of Physics Conference Series, 2019(1). https://doi.org/10.1088/1742-6596/2019/1/012016
Nabilah, F. N., Maknun, J., Muslim, M., Samsudin, A., Hasanah, L., & Suhandi, A. (2019). Eleventh-grade student’s conceptions about temperature and heat. Journal of Physics Conference Series, 1280(5). https://doi.org/10.1088/1742-6596/1280/5/052067
Oktaviani, N., & Istiyono, E. (2023). Application of partial model in practical performance assesment of physics learning. AIP Conference Proceedings, 2556. https://doi.org/10.1063/5.0130724
Ozdemir, B., & Alshamrani, A. H. (2020). Examining the fairness of language tests across gender with IRT-based differential item and test functioning methods. International Journal of Learning Teaching and Educational Research, 19(6), 27–45. https://doi.org/10.26803/ijlter.19.6.2
Planinic, M., Boone, W. J., Susac, A., & Ivanjek, L. (2019). Rasch analysis in physics education research: Why measurement matters. Physical Review Physics Education Research, 15(2). https://doi.org/10.1103/PhysRevPhysEducRes.15.020111
Rohima, I. E., & Hasbiah, A. W. (2023). Rasch analysis of household waste reduction behavior: case study in sunten jaya, bandung barat regency. Journal of Community Based Environmental Engineering and Management, 7(2), 111–118. https://doi.org/10.23969/JCBEEM.V7I2.10383
Romine, W. L., Schaffer, D. L., & Barrow, L. (2015). Development and application of a novel rasch-based methodology for evaluating multi-tiered assessment Instruments: Validation and utilization of an undergraduate diagnostic test of the water cycle. International Journal of Science Education, 37(16), 2740–2768. https://doi.org/10.1080/09500693.2015.1105398
Salele, N., Khan, S. H., Hasan, M., & Ali, S. (2025). Advancing four-tier diagnostic assessments: a novel approach to mapping engineering students’ conceptual understanding in microwave engineering course. IEEE Access, 13, 59886–59910. https://doi.org/10.1109/ACCESS.2025.3555432
Samsudin, A., Azizah, N., Fratiwi, N. J., Suhandi, A., Irwandani, I., Nurtanto, M., Yusup, M., Supriyatman, S., Masrifah, M., Aminudin, A. H., & Costu, B. (2024). Development of DIGaKiT: identifying students’ alternative conceptions by Rasch analysis model. Journal of Education and Learning (EduLearn), 18(1), 128–139. https://doi.org/10.11591/EDULEARN.V18I1.20970
Saputra, A., & Tania, L. (2024). An Indonesian-version of the short attitude toward mathematics inventory: a voice from pre-service chemistry teacher. International Journal of Evaluation and Research in Education (IJERE), 13(3), 1908–1916. https://doi.org/10.11591/IJERE.V13I3.27718
Schooler, C. (1968). A Note of Extreme Caution on the Use of. Source: American Journal of Sociology, 74(3), 296–301.
Steyn, R., & de Bruin, G. P. (2020). An investigation of gender-based differences in assessment instruments: A test of measurement invariance. SA Journal of Industrial Psychology, 46. https://doi.org/10.4102/sajip.v46i0.1699
Sukarelawan, M. I., Sriyanto, S., Puspitasari, A. D., Sulisworo, D., & Hikmah, U. N. (2021). Four-Tier Heat and Temperature Diagnostic Test (4T-HTDT) to Identify Student Misconceptions. JIPFRI (Jurnal Inovasi Pendidikan Fisika Dan Riset Ilmiah), 5(1), 1–8. https://doi.org/10.30599/JIPFRI.V5I1.856
Walfridsson, U., Walfridsson, H., Middeldorp, M. E., Sanders, P., & Årestedt, K. (2022). Validation of the English version of the arrhythmia-specific questionnaire in tachycardia and arrhythmia (ASTA): a Rasch evaluation study. Journal of Patient Reported Outcomes, 6(1). https://doi.org/10.1186/s41687-022-00493-4
Wind, S. A. (2018). Using Guttman errors to explore rater fit in rater-mediated performance assessments. Methodological Innovations, 11(3). https://doi.org/10.1177/2059799118814396
Xiao, X., Xue, M., & Cheng, Y. (2023). Bayesian partial credit model and its applications in science education. In Contemporary Trends and Issues in Science Education (Vol. 57). https://doi.org/10.1007/978-3-031-28776-3_4
Yang, F., Lu, H., Liu, W., & Zhou, H. (2020). Understanding the contributions of microscopic heat transfer to thermal conductivities of liquid aldehydes and ketones by molecular dynamics simulation. Journal of Chemical Information and Modeling, 60(6), 3022–3029. https://doi.org/10.1021/acs.jcim.0c00184
Yu, X., Tang, Q., Qin, C., & Li, Y. (2024). Person-Fit in cognitive diagnostic assessment | 认知诊断评价中的被试拟合研究. Journal of Psychological Science, 47(3), 744–751. https://doi.org/10.16719/j.cnki.1671-6981.20240329
Refbacks
- There are currently no refbacks.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.