Development and Validation of a Multidimensional Gen-AI Competence Scale for Pre-Service Science Teachers
The rapid integration of generative artificial intelligence (Gen-AI) into education is transforming the competencies required of future teachers and creating a need for a robust multidimensional measure of Gen-AI competency. This study developed and validated a six-dimensional framework comprising output evaluation, use appropriateness, prompting & operation, technical integration, model literacy, and responsible practice among pre-service science teachers. Data were obtained from 426 biology, chemistry, and physics education students at Universitas Jambi, Indonesia. The responses were randomly split into EFA (n = 256; 60%) and CFA (n = 170; 40%) subsamples. An initial 84-item pool was evaluated through ten experts’ judgment using CVR and CVI, resulting in 63 content-valid items. The data were suitable for factor analysis (KMO = 0.972; Bartlett’s χ² = 20430.533; df = 1,953; p < 0.001), and EFA retained 33 items across six dimensions, with factor loadings of 0.502–0.811 and Cronbach’s alpha of 0.765–0.909. CFA supported the six-dimensional structure with acceptable fit (RMSEA = 0.063, CFI = 0.913, TLI = 0.904, SRMR = 0.052). The dimensions demonstrated satisfactory reliability and convergent validity (α = 0.754–0.909; CR = 0.752–0.910; AVE = 0.505–0.611), while HTMT values of 0.586–0.874 supported discriminant validity. The resulting measure provides an empirically supported basis for profiling Gen-AI competency and strengthening teachers’ preparation for responsible Gen-AI use.
Keywords: generative artificial intelligence, gen-AI competency, instrument validation, pre-service science teachers.
Acquah, B. Y. S., Arthur, F., Gyedu, F. O., Quayson, E., Boateng, E., Quaye, S. A., & Nortey, S. A. (2026). Preservice teachers’ artificial intelligence competence: A network and latent profile analyses. European Journal of Education, 61(3), e70735. https://doi.org/10.1111/ejed.70735
Akgun, S., & Greenhow, C. (2022). Artificial intelligence in education: Addressing ethical challenges in K-12 settings. AI and Ethics, 2(3), 431–440. https://doi.org/10.1007/s43681-021-00096-7
Annapureddy, R., Fornaroli, A., & Gatica-Perez, D. (2025). Generative AI Literacy: Twelve Defining Competencies. Digital Government: Research and Practice, 6(1), 1–21. https://doi.org/10.1145/3685680
Ayanwale, M. A., Adelana, O. P., Molefi, R. R., Adeeko, O., & Ishola, A. M. (2024). Examining artificial intelligence literacy among pre-service teachers for future classrooms. Computers and Education Open, 6, 100179. https://doi.org/10.1016/j.caeo.2024.100179
Celik, I. (2023). Towards intelligent-TPACK: An empirical study on teachers’ professional knowledge to ethically integrate artificial intelligence (AI)-based tools into education. Computers in Human Behavior, 138, 107468. https://doi.org/10.1016/j.chb.2022.107468
Chiu, T. K. F., Ahmad, Z., & Çoban, M. (2025). Development and validation of teacher artificial intelligence (AI) competence self-efficacy (TAICS) scale. Education and Information Technologies, 30, 6667–6685. https://doi.org/10.1007/s10639-024-13094-z
Chiu, T. K. F., Ahmad, Z., Ismailov, M., & Sanusi, I. T. (2024). What are artificial intelligence literacy and competency? A comprehensive framework to support them. Computers and Education Open, 6, 100171. https://doi.org/10.1016/j.caeo.2024.100171
Dahlkemper, M. N., Lahme, S. Z., & Klein, P. (2023). How do physics students evaluate artificial intelligence responses on comprehension questions? A study on the perceived scientific accuracy and linguistic quality of ChatGPT. Physical Review Physics Education Research, 19(1), 010142. https://doi.org/10.1103/PhysRevPhysEducRes.19.010142
Delcker, J., Heil, J., & Ifenthaler, D. (2025). Evidence-based development of an instrument for the assessment of teachers’ self-perceptions of their artificial intelligence competence. Education and Information Technologies, 30, 115–133. https://doi.org/10.1007/s11423-024-10418-1
Ding, A.-C. E., Shi, L., Yang, H., & Choi, I. (2024). Enhancing teacher AI literacy and integration through different types of cases in teacher professional development. Computers and Education Open, 6, 100178. https://doi.org/10.1016/j.caeo.2024.100178
Dziuban, C. D., & Shirkey, E. C. (1974). When is a correlation matrix appropriate for factor analysis? Some decision rules. Psychological Bulletin, 81(6), 358–361. https://doi.org/10.1037/h0036316
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272–299. https://doi.org/10.1037/1082-989X.4.3.272
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.1177/002224378101800104
Goretzko, D., Siemund, K., & Sterner, P. (2024). Evaluating Model Fit of Measurement Models in Confirmatory Factor Analysis. https://doi.org/10.1177/00131644231163813
Groskurth, K., Bluemke, M., & Lechner, C. M. (2024). Why we need to abandon fixed cutoffs for goodness-of-fit indices : An extensive simulation and possible solutions. Behavior Research Methods, 56(4), 3891–3914. https://doi.org/10.3758/s13428-023-02193-3
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis (8th ed.). Cengage.
Hair, J. F., Howard, M. C., & Nitzl, C. (2020). Assessing measurement model quality in PLS-SEM using confirmatory composite analysis. Journal of Business Research, 109, 101–110. https://doi.org/10.1016/j.jbusres.2019.11.069
Hair, J. F., Risher, J. J., Sarstedt, M., & Ringle, C. M. (2019). When to use and how to report the results of PLS-SEM. European Business Review, 31(1), 2–24. https://doi.org/10.1108/EBR-11-2018-0203
Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43, 115–135. https://doi.org/10.1007/s11747-014-0403-8
Holzmann, U., Anand, S., & Payumo, A. Y. (2025). The ChatGPT Fact-Check: Exploiting the limitations of generative AI to develop evidence-based reasoning skills in college science courses. Advances in Physiology Education, 49(1), 191–196. https://doi.org/10.1152/advan.00142.2024
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., & Liu, T. (2025). A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. ACM Transactions on Information Systems, 43(2), 1–55. https://doi.org/10.1145/3703155
Kaiser, H. F. (1974). An Index of Factorial Simplicity. Psychometrika, 39(1), 31–36. https://doi.org/10.1007/BF02291575
Kim, W. J. (2026). Teachers’ use of generative artificial intelligence for designing science lessons in support of environmental science agency. Research in Science Education, 56, 203–222. https://doi.org/10.1007/s11165-025-10262-0
Kline, R. B. (2023). Principles and practice of structural equation modeling (5th ed.). Guilford Press.
Kohnke, L., Zou, D., Ou, A. W., & Gu, M. M. (2025). Preparing future educators for AI-enhanced classrooms: Insights into AI literacy and integration. Computers and Education: Artificial Intelligence. https://doi.org/https://doi.org/10.1016/j.caeai.2025.100398
Lawshe, C. H. (1975). A Quantitative Approach to Content Validity. Personnel Psychology, 28(4), 563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x
Lee, L.-S. (2023). New Challenges of Teaching Materials in STEM-Related ESD/SDG Activities in Taiwan. Online Submission.
Lo, L. S. (2023). The art and science of prompt engineering: A new literacy in the information age. Internet Reference Services Quarterly, 27(4), 203–210. https://doi.org/10.1080/10875301.2023.2227621
Long, D., & Magerko, B. (2020). What is AI literacy? Competencies and design considerations. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–16. https://doi.org/10.1145/3313831.3376727
Malik, I. S. (2026). Mastering Content Validation: A Guide to Calculating the Content Validity Index (CVI). Educational Quest- An International Journal of Education and Applied Social Sciences, 17(1), 21–27. https://doi.org/10.30954/2230-7311.1.2026.3
Mikeladze, T., Meijer, P. C., & Verhoeff, R. P. (2024). A Comprehensive Exploration of Artificial Intelligence Competence Frameworks for Educators: A Critical Review. European Journal of Education. https://doi.org/10.1111/ejed.12663
Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017–1054. https://doi.org/10.1111/j.1467-9620.2006.00684.x
Moorhouse, B. L., Wan, Y., Wu, C., Kohnke, L., Ho, T. Y., & Kwong, T. (2024). Developing language teachers’ professional generative AI competence: An intervention study in an initial language teacher education course. System, 125, 103399. https://doi.org/10.1016/j.system.2024.103399
Moundridou, M., Matzakos, N., & Doukakis, S. (2024). Generative AI tools as educators’ assistants: Designing and implementing inquiry-based lesson plans. Computers and Education: Artificial Intelligence, 7, 100277. https://doi.org/10.1016/j.caeai.2024.100277
Ng, D. T. K., Leung, J. K. L., Chu, K. W. S., & Qiao, M. S. (2021). AI Literacy: Definition, Teaching, Evaluation, and Ethical Issues. Proceedings of the Association for Information Science and Technology, 58(1), 504–509. https://doi.org/10.1002/pra2.487
Ng, D. T. K., Wu, W., Leung, J. K. L., Chiu, T. K. F., & Chu, S. K. W. (2024). Design and validation of the AI literacy questionnaire: The affective, behavioural, cognitive and ethical approach. British Journal of Educational Technology, 55(3), 1082–1104. https://doi.org/https://doi.org/10.1111/bjet.13411
Nunnally, J. C. (1978). Psychometric theory (2nd ed.). McGraw-Hill.
Park, J., & Choo, S. (2025). Generative AI prompt engineering for educators: Practical strategies. Journal of Special Education Technology, 40(3). https://doi.org/10.1177/01626434241298954
Pérez-Montesdeoca, H., Rodríguez-Rodríguez, D., Stendardi, D., & Fernández-Sogorb, A. (2026). Design and validation of a questionnaire on teachers’ uses of generative artificial intelligence. Computers and Education Open, 10, 100332. https://doi.org/10.1016/j.caeo.2026.100332
Polit, D. F., Beck, C. T., & Owen, S. V. (2007). Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health, 30(4), 459–467. https://doi.org/10.1002/nur.20199
Qian, Y. (2025). Pedagogical applications of generative AI in higher education: A systematic review of the field. TechTrends, 69, 1105–1120. https://doi.org/10.1007/s11528-025-01100-1
Rachmawati, K., Schultz, T., & Cusack, L. (2017). Translation, adaptation, and psychometric testing of a tool for measuring nurses’ attitudes towards research in Indonesian primary health care. Nursing Open, 4(2), 96–107.
Ramnarain, U., Ogegbo, A. A., Penn, M., Ojetunde, S., & Mdlalose, N. (2025). Pre-service science teachers’ intention to use generative artificial intelligence in inquiry-based teaching. Journal of Science Education and Technology, 34, 1272–1285. https://doi.org/10.1007/s10956-024-10159-z
Shi, L. (2025). Assessing teachers’ generative artificial intelligence competencies: instrument development and validation. Education and Information Technologies, 30(16), 23365–23384. https://doi.org/10.1007/s10639-025-13684-5
Somabut, A., Tuamsuk, K., Lowatcharin, G., Traiyarach, S., & Kwangmuang, P. (2025). Preparing for the AI era: Science teachers’ readiness and professional development needs for generative AI integration in secondary education. Social Sciences & Humanities Open, 12, 102259. https://doi.org/10.1016/j.ssaho.2025.102259
Sperling, K., Stenberg, C.-J., McGrath, C., Åkerfeldt, A., Heintz, F., & Stenliden, L. (2024). In search of artificial intelligence (AI) literacy in teacher education: A scoping review. Computers and Education Open, 6, 100169. https://doi.org/10.1016/j.caeo.2024.100169
Sun, D., Ba, S., Cha, Y., Yu, J., Chiang, F.-K., Dai, H. M., & Lim, C.-P. (2026). Empowering university teachers in higher education: A generative AI-responsive competency framework. Computers and Education: Artificial Intelligence, 10, 100542. https://doi.org/10.1016/j.caeai.2026.100542
Tagare, D., Karki, T., & Yu, W. (2025). K-12 teachers’ ethical competencies for AI literacy: Insights from a systematic literature review. Computers & Education. https://doi.org/10.1016/j.compedu.2025.105203
Tang, K. (2024). Informing research on generative artificial intelligence from a language and literacy perspective: A meta‐synthesis of studies in science education. Science Education, 108(5), 1329–1355. https://doi.org/10.1002/sce.21875
Wang, Y., Derakhshan, A., & Ghiasvand, F. (2025). EFL teachers’ generative artificial intelligence (GenAI) literacy: A scale development and validation study. System, 133, 103791. https://doi.org/10.1016/j.system.2025.103791
Worthington, R. L., & Whittaker, T. A. (2006). Scale Development Research: A Content Analysis and Recommendations for Best Practices. The Counseling Psychologist, 34(6), 806–838. https://doi.org/10.1177/0011000006288127
Wu, H., Zeng, Y., Chen, Z., & Liu, F. (2025). GenAI competence is different from digital competence: Developing and validating the GenAI competence scale for second language teachers. Education and Information Technologies, 30(16), 22567–22591. https://doi.org/10.1007/s10639-025-13672-9
Yan, L., Greiff, S., Teuber, Z., & Gašević, D. (2024). Promises and challenges of generative artificial intelligence for human learning. Nature Human Behaviour, 8, 1839–1850. https://doi.org/10.1038/s41562-024-02004-5
Yuan, R., Wang, K., Wang, Z., & He, W. (2026). Generative artificial intelligence (GenAI) in K-12 schools: A systematic review of teacher motivation and engagement. Educational Research Review, 51, 100787. https://doi.org/10.1016/j.edurev.2026.100787
Yue, M., Jong, M. S.-Y., & Ng, D. T. K. (2024). Understanding K-12 teachers’ technological pedagogical content knowledge readiness and attitudes toward artificial intelligence education. Education and Information Technologies, 29, 19505–19536. https://doi.org/10.1007/s10639-024-12621-2
Zou, D., Xie, H., & Kohnke, L. (2025). Navigating the future: Establishing a framework for educators’ pedagogic artificial intelligence competence. European Journal of Education, 60(2), e70117. https://doi.org/10.1111/ejed.70117
Refbacks
- There are currently no refbacks.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.