ChatGPT and Google Gemini in EFL Education: A Qualitative Exploration of Pedagogical Efficacy among Indonesian Sophomores
DOI:
https://doi.org/10.33394/jollt.v13i1.9926Keywords:
Active Learning, ChatGPT, EFL Instruction, Google Bard, Qualitative MethodologyAbstract
As generative language models like ChatGPT and Google Gemini gain prominence in education, their efficacy in specific contexts, such as Indonesian English as a Foreign Language (EFL) instruction, still needs to be explored. This study investigates the pedagogical affordances and constraints of these models as perceived by Indonesian EFL sophomores, aiming to understand their contribution to active learning in language acquisition. Using a qualitative approach, we conducted open-ended questionnaires with 40 sophomore students from an Indonesian university's English department. Thematic content analysis was employed to analyse the data. Findings reveal that ChatGPT offers authentic conversational simulations and versatile content-based instruction, while Google Gemini's strength lies in its multilingual capabilities. However, limitations such as linguistic complexity and rigid conversational structures were also identified. The study suggests these models can enhance active learning experiences, particularly in conversational practice and interdisciplinary content exploration, though their efficacy depends on factors like learner proficiency and internet access. We conclude that integrating these models into EFL instruction requires careful consideration of their affordances and limitations. This study contributes culturally-specific insights to AI in education research, with implications for curriculum designers, educators, and policymakers in developing countries, emphasising the need for adaptive and inclusive approaches in AI-enhanced EFL education.
References
Abulibdeh, A., Zaidan, E., & Abulibdeh, R. (2024). Navigating the confluence of artificial intelligence and education for sustainable development in the era of Industry 4.0: Challenges, opportunities, and ethical dimensions. Journal of Cleaner Production, 140527. http://dx.doi.org/10.1016/j.jclepro.2023.140527
Ali, J. K. M., Shamsan, M. A. A. . ., Hezam, T. A., & Mohammed, A. A. Q. (2023). Impact of ChatGPT on Learning Motivation: Teachers and Students’ Voices. Journal of English Studies in Arabia Felix, 2(1), 41–49. https://doi.org/10.56540/jesaf.v2i1.51
Alzhanova, A., & Chaklikova, A. (2022). Multilingual Education: Development of Professional Foreign Language Communicative Competence of Students in a Digital Environment. International Journal of Web-Based Learning and Teaching Technologies (IJWLTT), 17(1), 1-13. https://doi.org/10.4018/IJWLTT.294572
Au Yeung, J., Kraljevic, Z., Luintel, A., Balston, A., Idowu, E., Dobson, R. J., & Teo, J. T. (2023). AI chatbots not yet ready for clinical use. Frontiers in digital health, 5, 1161098. https://doi.org/10.3389/fdgth.2023.1161098
Basaffar, F. M. (2017). The Effect of Implementing Some Generative Learning Model Strategies in Teaching Reading Comprehension. International Journal of English Language Education, 5(1), 42-53. http://dx.doi.org/10.5296/ijele.v5i1.10612
Baskara, F. R. (2023). Exploring the implications of ChatGPT for language learning in higher education. Indonesian Journal of English Language Teaching and Applied Linguistics, 7(2), 343-358. http://dx.doi.org/10.21093/ijeltal.v7i2.1387
Benkhalfallah, F., Laouar, M. R., & Benkhalfallah, M. S. (2024, April). Empowering Education: Harnessing Artificial Intelligence for Adaptive E-Learning Excellence. In International Conference on Artificial Intelligence and its Applications in the Age of Digital Transformation (pp. 41-55). Cham: Springer Nature Switzerland. https://doi.org/10.18267/j.aip.240
Bentalha, B., & Alla, L. (2024). Revealing the subtleties: The art of qualitative studies in science and management. In Applying qualitative research methods to management science (pp. 1-21). IGI Global. https://doi.org/10.4018/979-8-3693-5543-5.ch001
Borji, A. (2023). A categorical archive of chatgpt failures. arXiv preprint arXiv:2302.03494. https://doi.org/10.48550/arXiv.2302.03494
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology, 3(2), 77-101. http://dx.doi.org/10.1191/1478088706qp063oa
Cancino, M., & Panes, J. (2021). The impact of Google Translate on L2 writing quality measures: Evidence from Chilean EFL high school learners. System, 98, 102464. https://doi.org/10.1016/j.system.2021.102464
Castleberry, A., & Nolen, A. (2018). Thematic analysis of qualitative research data: Is it as easy as it sounds?. Currents in pharmacy teaching and learning, 10(6), 807-815. https://doi.org/10.1016/j.cptl.2018.03.019
Cerf, V. G. (2019). Polyglot!. Communications of the ACM, 62(9), 6-6. https://doi.org/10.1145/3352690
Chang, Q., Pan, X., Manikandan, N., & Ramesh, S. (2022). Artificial intelligence technologies for teaching and learning in higher education. International Journal of Reliability, Quality and Safety Engineering, 29(05), 2240006. https://doi.org/10.1142/S021853932240006X
Chapelle, C. A., & Sauro, S. (2017). Introduction to the handbook of technology and second language teaching and learning. The handbook of technology and second language teaching and learning, 1-9. https://doi.org/10.1002/9781118914069
Chun, D., Kern, R., & Smith, B. (2016). Technology in language use, language teaching, and language learning. The Modern Language Journal, 100(S1), 64-80. https://doi.org/10.1111/modl.12302
Cohn, A. C., & Ravindranath, M. (2014). Local languages in Indonesia: Language maintenance or language shift. Linguistik Indonesia, 32(2), 131-148. https://doi.org/10.26499/li.v32i2.22
Crawford, J., Cowling, M., & Allen, K. A. (2023). Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). Journal of University Teaching & Learning Practice, 20(3), 02. http://dx.doi.org/10.53761/1.20.3.02
Creswell, J. W., & Poth, C. N. (2016). Qualitative inquiry and research design: Choosing among five approaches. Sage publications.
Dalkin, S., Forster, N., Hodgson, P., Lhussier, M., & Carr, S. M. (2021). Using computer assisted qualitative data analysis software (CAQDAS; NVivo) to assist in the complex process of realist theory generation, refinement and testing. International Journal of Social Research Methodology, 24(1), 123-134. https://doi.org/10.1080/13645579.2020.1803528
Dehalwar, K., & Sharma, S. N. (2024). Exploring the Distinctions between Quantitative and Qualitative Research Methods. Think India Journal, 27(1), 7-15. http://dx.doi.org/10.5281/zenodo.10553000
Dimitriadou, E., & Lanitis, A. (2023). A critical evaluation, challenges, and future perspectives of using artificial intelligence and emerging technologies in smart classrooms. Smart Learning Environments, 10(1), 12. https://doi.org/10.1186/s40561-023-00231-3
Dooly, M. (2009). New competencies in a new era? Examining the impact of a teacher training project. ReCALL, 21(3), 352-369. https://doi.org/10.1017/S0958344009990085
Duenas, M. (2004). The whats, whys, hows and whos of content-based instruction in second/foreign language education. International journal of English studies, 4(1), 73-96. Retrieved from https://revistas.um.es/ijes/article/view/48061
Ellis, N. C., & Larsen-Freeman, D. (2006). Language emergence: Implications for applied linguistics—Introduction to the special issue. Applied linguistics, 27(4), 558-589. https://psycnet.apa.org/doi/10.1093/applin/aml028
Etikan, I., Musa, S. A., & Alkassim, R. S. (2016). Comparison of convenience sampling and purposive sampling. American journal of theoretical and applied statistics, 5(1), 1-4. http://dx.doi.org/10.11648/j.ajtas.20160501.11
Eysenbach, G. (2023). The role of ChatGPT, generative language models, and artificial intelligence in medical education: a conversation with ChatGPT and a call for papers. JMIR Medical Education, 9(1), e46885. https://doi.org/10.2196/46885
Fiorella, L., & Mayer, R. E. (2016). Eight ways to promote generative learning. Educational psychology review, 28, 717-741. https://doi.org/10.1007/s10648-015-9348-9
Garcia, I., & Pena, M. I. (2011). Machine translation-assisted language learning: writing for beginners. Computer Assisted Language Learning, 24(5), 471-487. http://dx.doi.org/10.1080/09588221.2011.582687
Gay, G. (2018). Culturally responsive teaching: Theory, research, and practice. teachers college press.
Godwin-Jones, R. (2019). Riding the digital wilds: Learner autonomy and informal language learning. Language Learning & Technology, 23(1), 8–25. https://doi.org/10125/44667
Gozalo-Brizuela, R., & Garrido-Merchan, E. C. (2023). ChatGPT is not all you need. A State of the Art Review of large Generative AI models. arXiv preprint arXiv:2301.04655. https://doi.org/10.48550/arXiv.2301.04655
Guettala, M., Bourekkache, S., Kazar, O., & Harous, S. (2024). Generative artificial intelligence in education: Advancing adaptive and personalized learning. Acta Informatica Pragensia, 13(3), 460-489. http://dx.doi.org/10.18267/j.aip.235
Hatch, J. A. (2023). Doing qualitative research in education settings. State university of New York press.
Hirosh, Z., & Degani, T. (2018). Direct and indirect effects of multilingualism on novel language learning: An integrative review. Psychonomic bulletin & review, 25, 892-916. https://doi.org/10.3758/s13423-017-1315-7
Holmes, W. (2019). Artificial intelligence in education: Promises and implications for teaching and learning. Center for Curriculum Redesign.
Jabbar, A., Li, X., & Omar, B. (2021). A survey on generative adversarial networks: Variants, applications, and training. ACM Computing Surveys (CSUR), 54(8), 1-49. https://doi.org/10.48550/arXiv.2006.05132
Komara, C., & Tiarsiwi, F. (2021). Exploring Indonesian EFL learners’ perception of English learning grammar. Journal of English Language Teaching and Linguistics, 6(2), 459-470. http://dx.doi.org/10.21462/jeltl.v6i2.564
Krashen, S. (1985). The input hypothesis: Issues and implications. New York: Longman.
Lichtman, M. (2023). Qualitative research in education: A user's guide. Routledge.
Lim, W. M. (2024). What is qualitative research? An overview and guidelines. Australasian Marketing Journal, 14413582241264619. https://doi.org/10.1177/14413582241264619
Liyanage, I., & Tao, W. (2020). Preparation of teachers and multilingual education: Ethical, just, and student-focussed practices. Multilingual education yearbook 2020: Teacher education and multilingual contexts, 1-22. http://dx.doi.org/10.1007/978-3-030-41211-1_1
Maxwell, J. A. (2013). Qualitative research design: An interactive approach: An interactive approach. sage.
Megahed, F. M., Chen, Y. J., Ferris, J. A., Knoth, S., & Jones-Farmer, L. A. (2024). How generative AI models such as ChatGPT can be (mis) used in SPC practice, education, and research? An exploratory study. Quality Engineering, 36(2), 287-315. https://doi.org/10.1080/08982112.2023.2206479
Murphy, J. M. (2014). Intelligible, comprehensible, non-native models in ESL/EFL pronunciation teaching. System, 42, 258-269. https://doi.org/10.1016/j.system.2013.12.007
Nuankaew, P. (2022). Self-regulated learning model in educational data mining. International Journal of Emerging Technologies in Learning (iJET), 17(17), 4-27. https://doi.org/10.3991/ijet.v17i17.23623
Odugu, D. I. (2017). Linguistic Diversity and Education. In Re-thinking Postcolonial Education in Sub-Saharan Africa in the 21st Century (pp. 125-140). Brill. https://doi.org/10.1007/978-94-6300-962-1
Okal, B. O. (2014). Benefits of multilingualism in education. Universal Journal of Educational Research, 2(3), 223-229. http://dx.doi.org/10.13189/ujer.2014.020304
Ortega-MartÃn, M., GarcÃa-Sierra, Ó., Ardoiz, A., Ãlvarez, J., Armenteros, J. C., & Alonso, A. (2023). Linguistic ambiguity analysis in ChatGPT. arXiv preprint arXiv:2302.06426. https://doi.org/10.48550/arXiv.2302.06426
Ouyang, F., & Jiao, P. (2021). Artificial intelligence in education: The three paradigms. Computers and Education: Artificial Intelligence, 2, 100020. https://doi.org/10.1016/j.caeai.2021.100020
Paek, S., & Kim, N. (2021). Analysis of worldwide research trends on the impact of artificial intelligence in education. Sustainability, 13(14), 7941. https://doi.org/10.3390/su13147941
Pavlik, J. V. (2023). Collaborating with ChatGPT: Considering the implications of generative artificial intelligence for journalism and media education. Journalism & mass communication educator, 78(1), 84-93. https://doi.org/10.1177/10776958221149577
Piccardo, E. (2013). Plurilingualism and curriculum design: Toward a synergic vision. Tesol Quarterly, 47(3), 600-614. https://doi.org/10.1002/tesq.110
Pietilä, A. M., Nurmi, S. M., Halkoaho, A., & Kyngäs, H. (2020). Qualitative research: Ethical considerations. The application of content analysis in nursing science research, 49-69. http://dx.doi.org/10.1007/978-3-030-30199-6_6
Rana, S. (2023). AI and GPT for management scholars and practitioners: Guidelines and implications. FIIB Business Review, 12(1), 7-9. https://doi.org/10.1177/23197145231161408
Rastogi, A., Zang, X., Sunkara, S., Gupta, R., & Khaitan, P. (2020, April). Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset. In Proceedings of the AAAI conference on artificial intelligence (Vol. 34, No. 05, pp. 8689-8696). https://doi.org/10.1609/aaai.v34i05.6394
Ree, S., & Koh, Y. (2017). The Aims of Education in the Era of AI. Journal for History of Mathematics, 30(6), 341-351. http://dx.doi.org/10.14477/jhm.2017.30.6.341
Robinson, O. C. (2014). Sampling in interview-based qualitative research: A theoretical and practical guide. Qualitative research in psychology, 11(1), 25-41. https://doi.org/10.1080/14780887.2013.801543
Rohiyatussakinah, I. (2021). Implementation of MBKM and the Relationship of Curriculum Policy based on a Case of EFL Education in Japan. Journal of English Language Teaching and Literature (JELTL), 4(2), 39-50. https://doi.org/10.47080/jeltl.v4i2.1434
Sajja, R., Sermet, Y., Cikmaz, M., Cwiertny, D., & Demir, I. (2024). Artificial intelligence-enabled intelligent assistant for personalized and adaptive learning in higher education. Information, 15(10), 596. https://doi.org/10.3390/info15100596
Saputra, W. A. (2022). Analyzing Context in English as Lingua Franca (ELF): A Discourse Analysis in an Indonesian Higher Education Institution (HEI). IJOLEH: International Journal of Education and Humanities, 1(1), 18-29. https://doi.org/10.56314/ijoleh.v1i1.36
Selwyn, N. (2016). Is Technology Good for Education? Polity Press. http://au.wiley.com/WileyCDA/WileyTitle/productCd-0745696465.html
Soares, M. I. D. S., de MB Oliveira, F. C., Muniz, P. H., & do Nascimento, M. D. (2019, October). ANA: a virtual assistant that sees and hears to help tetraplegic online learning. In 2019 IEEE Frontiers in Education Conference (FIE) (pp. 1-5). Ieee. http://dx.doi.org/10.1109/FIE43999.2019.9028558
Strielkowski, W., Grebennikova, V., Lisovskiy, A., Rakhimova, G., & Vasileva, T. (2024). AIâ€driven adaptive learning for sustainable educational transformation. Sustainable Development. https://doi.org/10.1002/sd.3221
Stryker, S.B. and Leaver, B.L., Eds. (1997) Content-Based Instruction in Foreign Language Education: Models and Methods. Georgetown University Press, Washington DC.
Taghizadeh, M., & Hasani Yourdshahi, Z. (2020). Integrating technology into young learners' classes: language teachers' perceptions. Computer Assisted Language Learning, 33(8), 982-1006. http://dx.doi.org/10.1080/09588221.2019.1618876
Tomlinson, C. A. (2014). The differentiated classroom: Responding to the needs of all learners. Ascd.
Topsakal, O., & Topsakal, E. (2022). Framework for a foreign language teaching software for children utilizing AR, voicebots and ChatGPT (large language models). The Journal of Cognitive Systems, 7(2), 33-38. http://dx.doi.org/10.52876/jcs.1227392
Underwood, J. (2021). Speaking to Machines: Motivating Speaking through Oral Interaction with Intelligent Assistants. Research-publishing. net. http://dx.doi.org/10.14705/rpnet.2021.50.1247
Vaismoradi, M., & Snelgrove, S. (2019). Theme in Qualitative Content Analysis and Thematic Analysis. Forum Qualitative Sozialforschung Forum: Qualitative Social Research, 20(3). https://doi.org/10.17169/fqs-20.3.3376
Wallace, R. M. (2004). A framework for understanding teaching with the Internet. American educational research journal, 41(2), 447-488. Retrieved from https://www.learntechlib.org/p/98928/.
Warschauer, M., & Healey, D. (1998). Computers and language learning: An overview. Language teaching, 31(2), 57-71. https://doi.org/10.1017/S0261444800012970
Yang, Z. G., & Laki, L. J. (2023). Solving Hungarian natural language processing tasks with multilingual generative models. In Annales Mathematicae et Informaticae (Vol. 57, pp. 92-106). http://dx.doi.org/10.33039/ami.2022.11.001
Zuccon, G., & Koopman, B. (2023). Dr ChatGPT, tell me what I want to hear: How prompt knowledge impacts health answer correctness. arXiv preprint arXiv:2302.13793. https://doi.org/10.48550/arXiv.2302.13793
Downloads
Published
How to Cite
Issue
Section
Citation Check
License
License and Publishing Agreement
In submitting the manuscript to the journal, the authors certify that:
- They are authorized by their co-authors to enter into these arrangements.
- The work described has not been formally published before, except in the form of an abstract or as part of a published lecture, review, thesis, or overlay journal.
- That it is not under consideration for publication elsewhere,
- That its publication has been approved by all the author(s) and by the responsible authorities – tacitly or explicitly – of the institutes where the work has been carried out.
- They secure the right to reproduce any material that has already been published or copyrighted elsewhere.
- They agree to the following license and publishing agreement.
Copyright
Authors who publish with JOLLT Journal of Languages and Language Teaching agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (CC BY-SA 4.0) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.Â
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
Licensing for Data Publication
- Open Data Commons Attribution License, http://www.opendatacommons.org/licenses/by/1.0/ (default)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
