Download PDF : 
Assessing Instrument Reliability and Research Feasibility for EFL Learner Autonomy: Evidence from a Pilot Study in Xi’an, China
Cao Xiaojun
Akademi Pengajian Bahasa, Universiti Teknologi MARA, Shah Alam, Malaysia
Wan Nazihah Wan Mohamed
Xi’an Traffic Engineering Institute, Xi’an, China
Muhammad Saiful Anuar Yusoff
Akademi Pengajian Bahasa, Universiti Teknologi MARA Cawangan Kelantan, Malaysia
Abstract
It is important that the instruction of English as a Foreign Language (EFL) requires an understanding of how learning styles, learning engagement, and learner autonomy interact. Valid instruments are prerequisites for the research, especially in Chinese cultural contexts. The purpose of this pilot study is to evaluate the feasibility of research methods and validate the psychometric qualities of three modified scales that measure learning styles, learning engagement, and learner autonomy among non-English major undergraduates in Xi’an, China. The current study used a quantitative cross-sectional design and convenience sampling. One hundred undergraduates completed the online survey via the WJX platform. Recruitment rates, completion times, and data quality evaluated the study feasibility. The instrument reliability was evaluated by Cronbach’s alpha. Bilingual professionals examined the translated instruments’ cultural appropriateness. With a 71.4% response rate, a mean completion time of 12.4 minutes, and no missing data, the recruitment and response procedures were practical. All the Cronbach’s alpha values for learner autonomy, learning styles, and learning engagement exceeded 0.90, showing the instruments have outstanding internal consistency. The results validate the high reliability and cultural appropriateness of the instruments for the target population, as well as the suggested research methods. This pilot study offers a strong methodological basis for a more thorough examination of the connections between these constructs in the context of Chinese EFL.
Keywords: Learning styles, Learning engagement, Learner autonomy, Non-English major undergraduates
DOI: https://doi.org/10.24191/ejssh.v10i1.10122
References:
Al-Hoorie, A. H. (2017). Sixty years of language motivation research: Looking back and looking forward. SAGE Open, 7(1). https://doi.org/10.1177/2158244017701976
Althubaiti, S. M., Alharbi, N. S., Althubaiti, A., Alzahrani, A., & Agha, S. (2025). Locus of control, learning styles, and academic achievement of Saudi pre-professional medical students: A cross-sectional study. Academic Psychiatry, 49(1), 65-69. https://doi.org/10.1007/s40596-024-02056-9
Chung, J., & Kim, T. (2025). Leveraging large language models for enhanced back-translation: Techniques and applications. IEEE Access, 13, 61322 61328. https://doi.org/10.1109/ACCESS.2025.3557014
Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). SAGE Publications.
Cruchinho, P., López-Franco, M. D., Capelas, M. L., Almeida, S., Bennett, P. M., Miranda da Silva, M., ... & Gaspar, F. (2024). Translation, cross-cultural adaptation, and validation of measurement instruments: A practical guideline for novice researchers. Journal of Multidisciplinary Healthcare, 2701-2728. https://doi.org/10.2147/JMDH.S419714
Daflizar. (2023). Approaches to foster learner autonomy in EFL learning: A literature review. Journey: Journal of English Language and Pedagogy, 6(1), 148–160. https://doi.org/10.33503/journey.v6i1.621
Du, Y. (2024). A streamlined approach to scale adaptation: Enhancing validity and feasibility in educational measurement. Journal of Language Teaching, 4(3), 18-22. https://doi.org/10.54475/jlt.2024.017
Erdem-Kara, B. & Doğan, N. (2022). The effect of ratio of items indicating differential item functioning on computer adaptive and multi-stage tests. International Journal of Assessment Tools in Education, 9(3), 682–696. https://doi.org/10.21449/ijate.1105769
Foroutan, M., Nooreen, N., Gani, S. H., & Baki, R. (2013). The relationship between language learning autonomy extent and learning styles in Malaysian context. World Applied Sciences Journal, 24(3), 395-402. https://www.idosi.org/wasj/wasj24(3)2013.htm
Genç, G. (2015). Investigation of the relationships between autonomous foreign language learning, learning styles and some variables. Ankara University Journal of Faculty of Educational Sciences (JFES), 48(1), 129-160. https://doi.org/10.1501/Egifak_0000001356
George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference (4th ed.) Allyn & Bacon.
Hair Jr, J., Page, M., & Brunsveld, N. (2019a). Essentials of business research methods. Routledge.
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019b). Multivariate data analysis (8th ed.). Cengage Learning.
Hiver, P., Zhou, S. A., Tahmouresi, S., Sang, Y., & Papi, M. (2020). Why stories matter: Exploring learner engagement and metacognition through narratives of the L2 learning experience. System, 91, 102260. https://www.sciencedirect.com/science/article/abs/pii/S0346251X19306153
Holec, H. (1981). Autonomy and foreign language learning. Pergamon Press.
Kaiying, C. H. E. N., & Jing, C. A. I. (2022). A case study of student engagement in online EFL courses for young learners in China. Asian Journal of English Language Teaching, 31. https://cup.cuhk.edu.hk/image/catalog/journal/jpreview/AJELT31_65-108_full.pdf
Kim, D., Jesiek, B. K., & Mazzurco, A. (2022). Development and validation of the sojourn readiness assessment (SRA): Exploratory and confirmatory factor analysis and translation to Chinese. International Journal of Intercultural Relations, 88, 95–105. https://www.sciencedirect.com/science/article/abs/pii/S0147176722000451
Kırmızı, Ö., & Kıraç, K. (2018). A comparative study of learner autonomy in terms of gender and learning contexts. Atatürk Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 22(Özel Sayı 3), 2955-2967. https://dergipark.org.tr/en/download/article-file/622420
Kolb, A. Y., & Kolb, D. A. (2021). The Kolb experiential learning profile: A guide to experiential learning theory, KELP psychometrics and research on validity. Experience Based Learning Systems, LLC.
Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Prentice-Hall.
Kunselman, A. R. (2024). A brief overview of pilot studies and their sample size justification. Fertility and Sterility, 121(6), 899-901. https://www.sciencedirect.com/science/article/pii/S0015028224000815
Lam, R. C. K. (2015). Implementing assessment for learning in a Confucius context: the case of Hong Kong 2004-14. In D. Wyse, L. Hayward, & J. Pandya (Eds.), The Sage Handbook of Curriculum, Pedagogy and Assessment (Vol. 2, pp. 756-771). Sage Publications.
Le Masson, V. (2023). The magic number: How to optimise and improve your survey response rate. Kantar. https://www. kantar. com/inspiration/research-services/what-is-a-good-surveyresponse-rate-pf.
Maya, J., Luesia, J. F., & Pérez-Padilla, J. (2021). The relationship between learning styles and academic performance: Consistency among multiple assessment methods in psychology and education students. Sustainability, 13(6), 3341. https://www.mdpi.com/2071-1050/13/6/3341
Naidoo, N., Nguyen, V. T., Ravaud, P., Young, B., Amiel, P., Schanté, D., ... & Boutron, I. (2020). The research burden of randomized controlled trial participation: A systematic thematic synthesis of qualitative evidence. BMC Medicine, 18(1), 6. https://link.springer.com/article/10.1186/s12916-019-1476-5
Namaziandost, E., Çakmak, F., Heydarnejad, T., & Rezai, A. (2024). The predictive effects of learner autonomy and academic engagement on willingness to communicate, foreign language learning self-esteem, and L2 grit in an EFL context. Acta Psychologica, 250, 104528. https://www.sciencedirect.com/science/article/pii/S0001691824004062
Nematipour, M. (2012). A study of Iranian EFL learners’ autonomy level and its relationship with learning style. English Linguistics Research, 1(1), 126–136. https://doi.org/10.5430/elr.v1n1p126
Nunnally, J. C. (1978). Psychometric theory (2nd ed.). McGraw-Hill.
Peng, Y., Tan, L., Zhang, K., Zhu, N., Dong, H., & Gao, H. (2024). The mediating role of nutritional care literacy on the relationship between self-directed learning ability and nursing competence. BMC Nursing, 23(1), 513. https://doi.org/10.1186/s12912-024-02185-4
Pirola, F., Russo, E., Giombi, F., Di Maria, A., Baiardini, I., Spriano, G., & Malvezzi, L. (2026). Exploring patient-reported outcomes after lacrimal surgery in a large monocentric cohort: Cross-cultural adaptation of the Italian Lac-Q. Head & Face Medicine, 22, Article 15. https://doi.org/10.1186/s13005-026-00602-4
Renuse, Y. S. (2024). Understanding the significance of pilot testing in research and development. International Journal of Multidisciplinary Research & Reviews, 03(04), 8-13. https://doi.org/10.56815/IJMRR.V314.2024/8-13
Schaufeli, W. B., Martínez, I. M., Pinto, A. M., Salanova, M., & Bakker, A. B. (2002). Burnout and engagement in university students: A cross-national study. Journal of Cross-Cultural Psychology, 33(5), 464-481. https://doi.org/10.1177/0022022102033005003
Spratt, M., Humphreys, G., & Chan, V. (2002). Autonomy and motivation: Which comes first? Language Teaching Research, 6(3), 245–266. https://doi.org/10.1191/1362168802lr106oa
Sun, S. (2025). Exploring the relationship between learner autonomy and learner engagement among EFL students in Cambodian universities. Indonesian Journal of Education Research, 6(2), 187–197. https://doi.org/10.37251/ijoer.v6i2.1486
Tamimi, A., & Razeq, A. H.-A. (2020). Learning strategies role in developing autonomous EFL students. Dirasat: Educational Sciences, 47(4), 462–483. https://dsr.ju.edu.jo/djournals/index.php/Edu/article/view/2519
Tate, R., Beauregard, F., Peter, C., & Marotta, L. (2023). Pilot Testing as a Strategy to Develop Interview and Questionnaire Skills for Scholar Practitioners: A Selection of Education Doctorate Students’ Reflective Vignettes. Impacting Education: Journal on Transforming Professional Practice, 8(4), 20–25. https://doi.org/10.5195/ie.2023.333
Teresi, J. A., Yu, X., Stewart, A. L., & Hays, R. D. (2022). Guidelines for designing and evaluating feasibility pilot studies. Medical Care, 60(1), 95-103. https://doi.org/10.1097/MLR.0000000000001664
Treesattayanmunee, P., & Baharudin, S. M. (2024). Learner autonomy and interaction in English language learning among Thai EFL undergraduate students. LEARN Journal: Language Education and Acquisition Research Network, 17(1), 393–415. https://so04.tci-thaijo.org/index.php/LEARN/article/view/270392
Tsingos, C., & Smith, L. (2015). Learning styles: Preference for perceiving and processing information. Currents in Pharmacy Teaching and Learning, 7(5), 723–727. https://doi.org/10.1016/j.cptl.2015.06.014
Xu, J. F., Peng, R. Z., & Wu W. P. (2004). Survey and analysis of independent English learning ability of non-English majors [in Chinese]. Foreign Language Teaching and Research, 36(1), 64-68.
Wu, H. (2024). Modeling EFL learners’ engagement: Serial mediation of autonomy support, enjoyment, and ideal L2 self. Journal of English Education and Linguistics Studies, 12(2), Article 5409. https://doi.org/10.30762/jeels.v12i2.5409
Yin, Z., & Chew, R. S. Y. (2025). Fostering sustainable learning through AI-supported flipped classrooms: A conceptual framework for enhancing self-regulated learning in Chinese College English listening instruction. Journal of China-ASEAN Studies, 5(2), 45-55. https://so07.tci-thaijo.org/index.php/JCAS/article/view/7889
Zhu Jiang, J., Soo Kum Yoke, S., & Wan Zumusni Mustapha, W. (2025). Navigating learner autonomy in the digital age: Chinese college EFL students’ behaviour and challenges. International Journal of Modern Languages and Applied Linguistics, 9(4). https://doi.org/10.24191/ijmal.v9i4.6795
