The Trend of Multimodal Use in School Learning from The Past to The Future : A Systematic Literature Review

Authors

  • Dyah Worowirastri Ekowati Universitas Muhammadiyah Malang, Indonesia
  • Husamah Husamah Universitas Muhammadiyah Malang, Indonesia
  • Rina Wahyu Setyaningrum Universitas Muhammadiyah Malang, Indonesia
  • Puji Sumarsono Sultan Idris Education University, Malaysia

DOI:

https://doi.org/10.33394/jk.v10i4.12669

Keywords:

Multimodal, School Learning, Systematic Literature Review.

Abstract

This study aims to investigate the trends of multimodal use in school learning from the past to the future as an important reflection for education practitioners. This research used SLR method with 19 articles that met the criteria for analysis after inclusion and exclusion process used PRISMA. The result showed that a total of 19 articles conducted research using the most dominant approach, namely a qualitative approach of 73.68% followed by a mixed and quantitative approach. The main keywords that often appear using the VOSviewer application are multimodality, learning, school, and literacy. Related keywords that do not have a direct relationship with multimodality are digital media, media literacy, school, creativity, and cyberspace. This is an opportunity for future research. Regarding the country of origin of the authors, the most are from Australia, namely five articles. The results of the analysis using VOSviewer on the 19 articles showed that Neil Mercer had the most publications cited by other articles. The analysis of the 19 articles illustrates that the most frequent contribution is the development of learning materials, both virtual and non-virtual, using various media, ranging from manual to smartphones. The results of this study provide valuable insights and implications for current and future education policies, particularly in guiding the integration of multimodal learning approaches in schools. These findings offer a foundation for researchers and policymakers to explore and develop strategies aligned with diverse educational objectives and backgrounds..

Author Biography

Dyah Worowirastri Ekowati, Universitas Muhammadiyah Malang

Primay Education Department

References

Allahmoradi, M. (2018). A review about the concept of metaphorical aspects of annex buildings. Cogent Engineering, 5(1), 1–15. https://doi.org/10.1080/23311916.2018.1501865

Alvarez, A., Peña Teeters, L., Hamm-Rodríguez, M., & Dimidjian, S. (2021). Understanding children’s funds of identity as learners through multimodal self-expressions in Mexico City. Learning, Culture and Social Interaction, 29. https://doi.org/10.1016/j.lcsi.2021.100513

Barton, G., & Le, A. H. (2023). The importance of aesthetics for curriculum and job readiness: an exploration of student, teacher, and employer perceptions through appraisal theory (pp. 157–167).

Bezemer, J., & Kress, G. (2020). Semiotic work in the science classroom. Cultural Studies of Science Education, 15(1), 71–74. https://doi.org/10.1007/s11422-019-09957-4

Chen, S. (2016). Linguistic landscape and space: A multimodal analysis of linguistic landscape in robot open space. International Journal of Applied Linguistics and English Literature, 5(6), 90–98. https://doi.org/10.7575/aiac.ijalel.v.5n.6p.90

Chung, S.-J., & Huang, P.-S. (2022). Impact of Face-to-Face Interactions and Digital Learning Platforms on Learning Motivation, Learning Achievements and Peer Interaction in After-School Activities. Journal of Research in Education Sciences, 67(4), 1–34. https://doi.org/10.6209/JORIES.202212_67(4).0001

Creswell, J. W., & Clark, V. L. P. (2018). Designing and conducting mixed methods research approarch. 849.

Danielsson, K., & Selander, S. (2021). Multimodal Texts in Disciplinary Education. In Multimodal Texts in Disciplinary Education. https://doi.org/10.1007/978-3-030-63960-0

Debras, C. (2021). Multimodal profiles of je (ne) sais pas in spoken French. Journal of Pragmatics, 182, 42–62. https://doi.org/10.1016/j.pragma.2021.05.018

Değer, K. C., Köker, İ. H., Destek, S., Toprak, H., Yapalak, Y., Gönültaş, C., & Şentürk, H. (2022). The clinical feature and outcome of groove pancreatitis in a cohort: A single center experience with review of the literature. Ulusal Travma ve Acil Cerrahi Dergisi, 28(8), 1186–1192. https://doi.org/10.14744/tjtes.2022.12893

DeJaynes, T. (2015). “Where I’m From†and belonging: A multimodal, cosmopolitan perspective on arts and inquiry. E-Learning and Digital Media, 12(2), 183–198. https://doi.org/10.1177/2042753014567236

Ekowati, D. W., Nusantara, T., Muksar, M., & Agus, D. (2024). A Literature Review Of Multimodal Semiotic Reasoning In Mathematics. Pegem Journal of Education and Instruction, 14(2), 261–274. https://doi.org/10.47750/pegegog.14.02.30

Ekowati, D. W., Nusantara, T., Muksar, M., Sudjimat, D. A., Yayuk, E., & Ismail, A. D. (2023). Semiotic Reasoning in Mathematics: A Systematic Literature Review of Future Trends and Opportunities. Migration Letters, 20(5), 971–991. https://doi.org/10.47059/ml.v20i5.4158

Ekowati, D. W., & Suwandayani, B. istanti. (2021). Learning from Home Strategies in Private Elementary Schools During Covid-19 Pandemic. International Journal of Elementary Education, 5(2), 375. https://doi.org/10.23887/ijee.v5i3.33548

Exley, B., & Mills, K. A. (2012). Parsing the Australian English curriculum: Grammar, multimodality and cross-cultural texts. Australian Journal of Language and Literacy, 35(2), 192–205. https://doi.org/10.1007/bf03651882

Fadeev, A. (2020). Acquisition of artistic literacy in multimodal learning via intersemiotic translation. Punctum International Journal of Semiotics, 6(1), 133–159. https://doi.org/10.18680/hss.2020.0007

Gordon, C. S., Howard, S. J., Jones, S. C., & Kervin, L. K. (2016). Evaluation of an Australian alcohol media literacy program. Journal of Studies on Alcohol and Drugs, 77(6), 950–957. https://doi.org/10.15288/jsad.2016.77.950

Hennessy, S., Howe, C., Mercer, N., & Vrikki, M. (2020). Coding classroom dialogue: Methodological considerations for researchers. Learning, Culture and Social Interaction, 25(January), 100404. https://doi.org/10.1016/j.lcsi.2020.100404

Kachorsky, D., Reid, S. F., & Chapman, K. (2020). Education Through TIME: Representations of U.S. Education on TIME Magazine Covers. AERA Open, 6(3), 1–19. https://doi.org/10.1177/2332858420961110

Kajamaa, A., & Kumpulainen, K. (2020). Students’ multimodal knowledge practices in a makerspace learning environment. International Journal of Computer-Supported Collaborative Learning, 15(4), 411–444. https://doi.org/10.1007/s11412-020-09337-z

Ke, F., & M. Clark, K. (2020). Game-Based Multimodal Representations and Mathematical Problem Solving. International Journal of Science and Mathematics Education, 18(1), 103–122. https://doi.org/10.1007/s10763-018-9938-3

Kozhemyakin, E. A., & Lovyagina, V. G. (2020). Targeted advertising in social networks: Searching for efficient semiotic models. Vestnik Moskovskogo Universiteta. Seriya 10. Zhurnalistika, 2020(5), 3–28. https://doi.org/10.30547/vestnik.journ.5.2020.328

Liu, P., Qiu, X., & Huang, X. (2017). Adversarial multi-task learning for text classification. ACL 2017 - 55th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers), 1, 1–10. https://doi.org/10.18653/v1/P17-1001

Lustig, A., Brookes, G., & Hunt, D. (2021). Social Semiotics of Gangstalking Evidence Videos on YouTube: Multimodal Discourse Analysis of a Novel Persecutory Belief System. JMIR Mental Health, 8(10). https://doi.org/10.2196/30311

Mabena, N., Mokgosi, P. N., & Ramapela, S. S. (2021). Factors Contributing To Poor Learner Performance in Mathematics: a Case of Selected Schools in Mpumalanga Province, South Africa. Problems of Education in the 21st Century, 79(3), 451–466. https://doi.org/10.33225/pec/21.79.451

Mahler, D., Großschedl, J., & Harms, U. (2018). Does motivation matter? – The relationship between teachers’ self-efficacy and enthusiasm and students’ performance. PLoS ONE, 13(11), 1–19. https://doi.org/10.1371/journal.pone.0207252

Maulida, F., Fitriani, A. D., & Darmayanti, M. (2024). Development of Teaching Materials Based on Differentiated Learning to Improve Critical Thinking Dimensions of The Pancasila Learner Profile. Jurnal Kependidikan: Jurnal Hasil Penelitian Dan Kajian Kepustakaan Di Bidang Pendidikan, Pengajaran Dan Pembelajaran, 10(1), 125. https://doi.org/10.33394/jk.v10i1.10420

Mills, K A, & Brown, A. (2022). Immersive virtual reality (VR) for digital media making: transmediation is key. Learning, Media and Technology, 47(2), 179–200. https://doi.org/10.1080/17439884.2021.1952428

Mills, K A, & Doyle, K. (2019). Visual arts: a multimodal language for Indigenous education. Language and Education, 33(6), 521–543. https://doi.org/10.1080/09500782.2019.1635618

Mills, Kathy A. (2010). “Filming in Progressâ€: New spaces for multimodal designing. Linguistics and Education, 21(1), 14–28. https://doi.org/10.1016/j.linged.2009.12.003

Moreno-Morilla, C., Guzmán-Simón, F., & García-Jiménez, E. (2021). Learning ecologies in the digital era: A challenge for primary education in low-income contexts. Mathematics, 9(17). https://doi.org/10.3390/math9172108

Papadopoulou, A. (2019). Art, technology, education: Synergy of modes, means, tools of communication. Education Sciences, 9(3). https://doi.org/10.3390/educsci9030237

Pazaer, N., & Assaiqeli, A. (2023). Pre-service Teachers’ Views on Using English Comics to Improve Pupils’ English Language Skills. Studies in English Language and Education, 10(1), 197–215. https://doi.org/10.24815/siele.v10i1.21836

Rickmann, A., Al-Nawaiseh, S., Ramirez, L., Röhrig, S., Ladewig, M., Szurman, P., & Szurman, G. (2020). Progressive maculopathy despite discontinuation of chloroquine treatment—multimodal imaging and review of the literature. Ophthalmologe, 117(9), 917–925. https://doi.org/10.1007/s00347-019-00994-6

Shazly, R. E. (2021). Splits on Instagram: A case study of young adults’ selfies. Semiotica, 2021(241), 185–218. https://doi.org/10.1515/sem-2019-0099

Taylor, L., & Clarke, P. (2021). We read, we write: reconsidering reading–writing relationships in primary school children. Literacy, 55(1), 14–24. https://doi.org/10.1111/lit.12235

Tytler, R., Prain, V., Kirk, M., Mulligan, J., Nielsen, C., Speldewinde, C., White, P., & Xu, L. (2023). Characterising a Representation Construction Pedagogy for Integrating Science and Mathematics in the Primary School. International Journal of Science and Mathematics Education, 21(4), 1153–1175. https://doi.org/10.1007/s10763-022-10284-4

ul Sabah, N., Sharif, H., & Khan, M. H. (2020). Persuasive strategies and video games: An insight into age of empire III. International Journal of Media and Information Literacy, 5(2), 191–198. https://doi.org/10.13187/IJMIL.2020.2.191

Unsworth, L. (2021). High school science infographics: Multimodal meaning complexes in composite image-language ensembles. Pensamiento Educativo, 58(2). https://doi.org/10.7764/PEL.58.2.2021.9

Wang, T., & Tan, L. (2023). The Conceptualisation of User-App Interactivity in Augmented Reality-Mediated Learning: Implications for Literacy Education. Sustainability (Switzerland), 15(14), 1–20. https://doi.org/10.3390/su151410949

Wernholm, M., & Reneland-Forsman, L. (2019). Children’s representation of self in social media communities. Learning, Culture and Social Interaction, 23.

Wu, X., & Fitzgerald, R. (2021). Reaching for the stars: DingTalk and the Multi-platform creativity of a ‘one-star’ campaign on Chinese social media. Discourse, Context and Media, 44. https://doi.org/10.1016/j.dcm.2021.100540

Zarate, A., & Medina, P. (2021). Teenagers and multimodal digital writing in a written communication course during the COVID-19 pandemic. 1st IEEE International Conference on Advanced Learning Technologies on Education and Research, ICALTER 2021. https://doi.org/10.1109/ICALTER54105.2021.9675129

Zhang, L. T., & Cassany, D. (2021). “The murderer is him √â€: Multimodal humor in danmu video comments. Internet Pragmatics, 4(2), 272–294. https://doi.org/10.1075/ip.00038.zha

Zuhri, R. S., Wilujeng, I., Haryanto, H., & Ibda, H. (2024). Information communication technologies education in elementary school: a systematic literature review. Journal of Education and Learning (EduLearn), 18(3), 1070–1082. https://doi.org/10.11591/edulearn.v18i3.21435

Zwitserlood, R., Wijnen, F., Van Weerdenburg, M., & Verhoeven, L. (2015). “MetaTaalâ€: Enhancing complex syntax in children with specific language impairment-a metalinguistic and multimodal approach. International Journal of Language and Communication Disorders, 50(3), 273–297. https://doi.org/10.1111/1460-6984.12131

Downloads

Additional Files

Published

2024-12-21

How to Cite

Ekowati, D. W., Husamah, H., Setyaningrum, R. W., & Sumarsono, P. (2024). The Trend of Multimodal Use in School Learning from The Past to The Future : A Systematic Literature Review. Jurnal Kependidikan, 10(4), 1548–1561. https://doi.org/10.33394/jk.v10i4.12669

Issue

Vol. 10 No. 4 (2024): December

Section

Articles

Citation Check

License

License and Publishing Agreement
In submitting the manuscript to the journal, the authors certify that:

  • They are authorized by their co-authors to enter into these arrangements.
  • The work described has not been formally published before, except in the form of an abstract or as part of a published lecture, review, thesis, or overlay journal.
  • That it is not under consideration for publication elsewhere,
  • That its publication has been approved by all the author(s) and by the responsible authorities tacitly or explicitly of the institutes where the work has been carried out.
  • They secure the right to reproduce any material that has already been published or copyrighted elsewhere.
  • They agree to the following license and publishing agreement.

Copyright
Authors who publish with JK agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (CC BY-SA 4.0) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal. 
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.

Licensing for Data Publication

JK use a variety of waivers and licenses, that are specifically designed for and appropriate for the treatment of data:
Other data publishing licenses may be allowed as exceptions (subject to approval by the editor on a case-by-case basis) and should be justified with a written statement from the author, which will be published with the article.

 

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.