Development of Algebra Test Using the Item Response Theory Approach for Junior High School Students

Authors

  • Ahmad Rustam Universitas Sulawesi Tenggara, Indonesia
  • Ode Zulaeha Institut Sains dan Kependidikan (ISDIK) Kie Raha Maluku Utara, Indonesia
  • Wiwin Rita Sari Universitas Nahdlatul Ulama Lampung, Indonesia
  • Suciati Rahayu Widyastuti Universitas Nahdlatul Ulama Cirebon, Indonesia

DOI:

https://doi.org/10.33394/jk.v10i3.11832

Keywords:

Test Development, Algebra Material, Item Response Theory, Two Logistic Parameters.

Abstract

This research aims to develop valid and reliable measuring tool for students' algebraic abilities that can be used in schools and the general public. The research follows a structured test development design, including stages such as preparing test specifications and items, field testing, revising items, and test development. The questions are aligned with the 2013 curriculum syllabus, ensuring relevance to educational standards. The test was given to 662 junior high school students in Kendari City, Indonesia, and their responses were analyzed using the item response theory (IRT) model with two logistic parameters: item difficulty level and item discriminatory power. The BILOG MG program was employed to estimate item and ability parameters. Before conducting item analysis with IRT, essential assumption tests were conducted, including unidimensional and model fit tests. The results of the development process, based on item analysis using BILOG MG, yielded 15 items covering various aspects of algebraic abilities. These items were derived from indicators such as recognizing algebraic forms, identifying elements within these forms, performing addition, subtraction, multiplication, and division operations on algebraic forms, presenting and solving real-world problems in algebraic contexts, and addressing contextual problems involving algebraic operations. The items demonstrated good fit with the model and exhibited an appropriate level of item difficulty and discriminatory power, making them suitable for use as a reliable assessment tool. Consequently, these developed tests are deemed effective for measuring students' foundational algebraic abilities.

References

Alkursheh, T. O., Al-zboon, H. S., & AlNasraween, M. S. (2022). The Effect of Item Form on Estimating Person ’ s Ability , Item Parameters , and Information Function According to Item Response Theory ( IRT ). International Journal of Instruction, 15(3), 1111–1130.

Balasubramanian, B. A, Cohen, D. J, Davis, M. M, Gunn, R, Dickinson, L. M, Miller, W. L, ... & Stange, K. C. (2015). Learning evaluation: blending quality improvement and implementation research methods to study healthcare innovations. Implementation Science, 10(1), 1–11.

Brown, T. A., & Moore, M. T. (2012). Confirmatory factor analysis. In Handbook of structural equation modeling (p. 361,379).

Brown, J. D. (2013). Classical test theory (pp. 337–349). n The Routledge handbook of language testing.

Budiyono. (2009). The Accuracy of Mantel-Haenszel, Sibstest, and Logistic regression Methods in Differential Item Functioning Detection. Jurnal Penelitian Dan Evaluasi Pendidikan, 12(1), 1–20.

Davier, M. Von, Yamamoto, K., Shin, H. J., Chen, H., & Khorramdel, L. (2019). Evaluating item response theory linking and model fit for data from PISA 2000–2012. Assessment in Education: Principles, Policy & Practice, 26(4), 466–488.

Elken, M. (2015). Developing policy instruments for education in the EU: The European qualifications framework for lifelong learning. International Journal of Lifelong Education, 34(6), 710–726.

Embretson, S. E., & Reise, S. P. (2013). Item response theory. Psychology Press.

Hakim, M. L., Muslim, & Ramalis, T. R. (2019). Karakteristik Tes Hasil Belajar Ranah Kognitif Materi Elastisitas Menggunakan Analisis Item Response Theory. Jurnal Penelitian Pembelajaran Fisika, 10(1), 22–32.

Hambleton, R. K., & Jones, R. W. (1993). Comparison of Classical Test Theory and Item Response Theory and Their Applications to Test Development. Educational Measurement, 38–47.

Hambleton, R. K., & Swaminathan, H. (1985). Item Response Theory Principles and Applications. Springer Science+Business Media, LLC.

Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of Item Response Theory. SAGE Publications Inc.

Hox, J. J. (2021). Confirmatory factor analysis. The Encyclopedia of Research Methods in Criminology and Criminal Justice, 2, 830–832.

Huljannah, M. (2021). Pentingnya Proses Evaluasi dalam Pembelajaran di Sekolah Dasar. Directory of Elementary Education Journal, 2(2), 49–63.

Idrus, L. (2019). Evaluasi Dalam Proses Pembelajaran 1. Evaluasi Dalam Proses Pembelajaran, 9(2), 920–935.

Irvine, S. H., & Kyllonen, P. C. (2013). Item generation for test development . Routledge.

Jabrayilov, R, Emons, W. H, & Sijtsma, K. (2016). Comparison of classical test theory and item response theory in individual change assessment. Applied Psychological Measurement, 40(8), 559–572. file:///Users/andreataquez/Downloads/guia-plan-de-mejora-institucional.pdf%0Ahttp://salud.tabasco.gob.mx/content/revista%0Ahttp://www.revistaalad.com/pdfs/Guias_ALAD_11_Nov_2013.pdf%0Ahttp://dx.doi.org/10.15446/revfacmed.v66n3.60060.%0Ahttp://www.cenetec.

Kean, J., & Reilly, J. (2014a). Item response theory. In Handbook for clinical research: Design, statistics and implementation (pp. 195–198).

Kean, J., & Reilly, J. (2014b). Item response theory. In Handbook for clinical research: Design, statistics and implementation (pp. 195–198).

Kereh, C. T., Sabadar, J., & Tjiang, P. C. (2013). Identifikasi kesulitan belajar mahasiswa dalam konten matematika pada materi pendahuluan fisika inti. Proceedings of Seminar Nasional Sains Dan Pendidikan Sains VIII, Fakultas Sains Dan Matematika, UKSW Salatiga, 4(1), 11–12.

Lia, R. M, Rusilowati, A, & Isnaeni, W. (2020). NGSS-Oriented chemistry test instruments: validity and reliability analysis with the rasch model. REiD (Research and Evaluation in Education), 6(1), 41–50.

Lord, F. M., & Novick, M. R. (1968). Statistical Theories of Mental Test Scores. Reading, MA: Addison-Wesley.

Mahirah, B. (2017). Evaluasi belajar peserta didik (siswa). Idaarah: Jurnal Manajemen Pendidikan, 1(2).

Mardianto. (2012). Psikologi Pendidikan. Perdana Publishing.

Mohajan, H. (2017). Two criteria for good measurements in research: Validity and reliability. Annals of Spiru Haret University Economics Series, 17(4), 59–82.

Mohamad, M. M., Sulaiman, N. L., Sern, L. C., & Salleh, K. M. (2015). Measuring the validity and reliability of research instruments. Procedia-Social and Behavioral Sciences, 204, 164–171.

Muhsetyo, G., Krisnadi, E., & Wahyuningrum, E. (2014). Pembelajaran matematika SD. Universitas Terbuka.

Pyrczak, F. (1973). Validity of the Discrimination Index As A Measure of Item Quality. Journal of Educational Measurement, 10(3), 227–231.

Raykov, T., Dimitrov, D. M., Marcoulides, G. A., & Harrison, M. (2019). On true score evaluation using item response theory modeling. Educational and Psychological Measuremen, 79(4), 796–807.

Retnawati, H. (2013). Pendeteksian Keberfungsian Butir Pembeda dengan Indeks Volume Sederhana berdasarkan Teori Respons Butir Multidimensi. Jurnal Penelitian Dan Evaluasi Pendidikan, 17(2), 275–286.

Sarea, M. S., & Ruslan, R. (2019). “Karakteristik Butir Soal: Classical Test Theory vs Item Response Theory?.†Didaktika: Jurnal Kependidikan 13.1, 13(1), 1–16.

Sirodj, D. A. N. (2018). Analisis Kualitas Aitem Intelligence Structure Test (IST) melalui Metode Item Response Theory (IRT). Schema: Journal of Psychological Research, 4(2), 98–107.

Soedjadi, R. (1996). Diagnosis Kesulitan Siswa Sekolah Dasar dalam Belajar Matematika. Jurnal Jurusan Matematika FPMIPA IKIP Surabaya, 25–33.

Suardipa, I. P., & Primayana, K. H. (2020). Peran desain evaluasi pembelajaran untuk meningkatkan kualitas pembelajaran. Widyacarya: Jurnal Pendidikan, Agama Dan Budaya, 4(2), 88–100. http://mpoc.org.my/malaysian-palm-oil-industry/

Subali, B., Kumaidi, & Nonoh, S. A. (2021). The Comparison of Item Test Characteristics Viewed from Classic and Modern Test Theory. International Journal of Instruction, 14(1), 647–660.

Subali, B., Kumaidi, Nonoh, S. A., & Sumintono, B. (2019). Student achievement based on the use of scientific method in the natural science subject in elementary school. Jurnal Pendidikan IPA Indonesia, 8(1), 39–51.

Sugiri, W. A., & Priatmoko, S. (2020). Persprektif asesmen autentik sebagai alat evaluasi dalam merdeka belajar. At-Thullab: Jurnal Pendidikan Guru Madrasah Ibtidaiyah, 4(1), 53–61.

Yang, F. M. (2014). Item response theory for measurement validity. Shanghai Archives of Psychiatry, 26(3), 171.

Zimowski, M. F. (2017). BILOG-MG. In Handbook of Item Response Theory (pp. 435–446). Chapman and Hall/CRC.

Downloads

Additional Files

Published

2024-09-18

How to Cite

Rustam, A., Zulaeha, O., Sari, W. R., & Widyastuti, S. R. (2024). Development of Algebra Test Using the Item Response Theory Approach for Junior High School Students. Jurnal Kependidikan: Jurnal Hasil Penelitian Dan Kajian Kepustakaan Di Bidang Pendidikan, Pengajaran Dan Pembelajaran, 10(3), 847–858. https://doi.org/10.33394/jk.v10i3.11832

Issue

Vol. 10 No. 3 (2024): September

Section

Articles

Citation Check

License

License and Publishing Agreement
In submitting the manuscript to the journal, the authors certify that:
  • They are authorized by their co-authors to enter into these arrangements.
  • The work described has not been formally published before, except in the form of an abstract or as part of a published lecture, review, thesis, or overlay journal.
  • That it is not under consideration for publication elsewhere,
  • That its publication has been approved by all the author(s) and by the responsible authorities – tacitly or explicitly – of the institutes where the work has been carried out.
  • They secure the right to reproduce any material that has already been published or copyrighted elsewhere.
  • They agree to the following license and publishing agreement.
Copyright
Authors who publish with JK agree to the following terms:
  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (CC BY-SA 4.0) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal. 
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
Licensing for Data Publication
JK use a variety of waivers and licenses, that are specifically designed for and appropriate for the treatment of data:
Other data publishing licenses may be allowed as exceptions (subject to approval by the editor on a case-by-case basis) and should be justified with a written statement from the author, which will be published with the article.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.