Development of Critical Thinking Skills Instruments : Cases for Essay Tests

Authors

  • Nurkhairo Hidayati Biology Education Department, Universitas Islam Riau, Indonesia
  • Suryanti Suryanti Biology Education Department, Universitas Islam Riau, Indonesia
  • Restesa Rahmayumita Master Program in Science Education, Universitas Riau, Indonesia
  • Selvira Aisya Biology Education Department, Universitas Islam Riau, Indonesia

DOI:

https://doi.org/10.33394/jk.v10i1.10052

Keywords:

Critical Thinking, Rubric, Assessment.

Abstract

This research aims to develop a rubric used to assess students’ critical thinking. This rubric refers to aspects of Ennis' critical thinking. The research method begins with determining the scope of the rubric so that an initial version of the rubric can be prepared. An initial version of the rubric received peer review. Next, the rubric was tested to assess critical thinking tests with essay-type questions. Data analysis was carried out using Cohen's kappa, while aspects of the critical thinking rubric were analyzed descriptively. The research results show that each aspect of critical thinking has been equipped with descriptors that can be used by raters. The level of reliability based on the kappa value of agreement is quite good, which means that different raters can use the rubric consistently. The value of agreement in the basic clarification aspect is 0.33 (fair), bases for a decision (0.571 in the moderate category), inference (0.455, moderate category), advanced clarification (0.250, fair category, supposition and integration (0.250, fair category), and strategies and tactics (0.182, poor category). Then, it can be concluded that the critical thinking rubric developed can be used to assess critical thinking abilities. As a recommendation, of course, this rubric cannot fulfill all types of critical thinking questions tested. It is also necessary to develop rubrics for question types other than essays to accommodate the wider need for critical thinking rubrics.

References

Abrami, P. C., Bernard, R. M., Borokhovski, E., Waddington, D. I., Wade, C. A., & Persson, T. (2014). Strategies for Teaching Students to Think Critically: A Meta-Analysis. Review of Educational Research, 1–40. https://doi.org/10.3102/0034654314551063

Anderson, T., & Soden, R. (2001). Peer interaction and the learning of critical thinking skills. Psychology Learning & Teaching, 1(1), 37–40.

Bezanilla, M. J., Fernández-Nogueira, D., Poblete, M., & Galindo-Domínguez, H. (2019). Methodologies for teaching-learning critical thinking in higher education: The teacher’s view. Thinking Skills and Creativity, 33(100584), 1–10. https://doi.org/10.1016/j.tsc.2019.100584

Biggs, J. (2014). Constructive alignment in university teaching. HERDSA Review of Higher Education, 1(1), 5–22.

Brookhart, S. M., & Chen, F. (2015). The quality and effectiveness of descriptive rubrics. Educational Review, 67(3), 343–368. https://doi.org/10.1080/00131911.2014.929565

Butler, H. A. (2012). Halpern Critical Thinking Assessment Predicts Real-World Outcomes of Critical Thinking. Applied Cognitive Psychology, 26(5), 721–729. https://doi.org/10.1002/acp.2851

Button, C. M., Snook, B., & Grant, M. J. (2020). Inter-Rater Agreement, Data Reliability, and The Crisis of Con fi dence in Psychological Research. Quant Methods Psychol, 16(5), 467–471.

Cooke, N. A. (2017). Posttruth, Truthiness, and Alternative Facts: Information Behavior and Critical Information Consumption for a New Age. The Library Quarterly, 87(3), 211–221. https://doi.org/10.1086/692298

Cooper, K. M., & Brownell, S. (2020). Student Anxiety and Fear of Negative Evaluation in Active Learning Science Classrooms Chapter 56 Student anxiety and fear of negative evaluation in active learning science classrooms. February. https://doi.org/10.1007/978-3-030-33600-4

Din, M. (2020). Evaluating university students’ critical thinking ability as re fl ected in their critical reading skill : A study at bachelor level in Pakistan. Thinking Skills and Creativity, 35(100627), 1–11. https://doi.org/10.1016/j.tsc.2020.100627

Ennis, R. H. (2011). The Nature of Critical Thinking: An Outline of Critical Thinking Dispositions. 1–8. http://faculty.education.illinois.edurhennis/documents. TheNatureofCriticalThinking_51711_000.pdf

Facione, P. A. (2000). The Disposition Toward Critical Thinking: Its Character, Measurement, and Relationship to Critical Thinking Skill. Informal Logic, 20(1), 61–84.

Golter, P., Brown, G., & Wie, B. Van. (2016). Preliminary Validation of a Critical Thinking Rubric for Fluid Mechanics and Heat Transfer. Journal of STEM Education, 17(4), 19–28.

Gray, K., & Koncz, A. (2018). The key attributes employers seek on students’ resumes. http://www.naceweb.org/about-us/press/2017/the- key-attributes-employers-seek-on-students-resumes/

Halpern, D. F. (2010). Halpern Critical Thinking Assessment. SCHUHFRIED. http://www.schuhfried.com/vienna-test-system-vts/all-tests-from-a-z/test/hcta-halpern-critical-thinking-assessment-1/

Hart, C., Da, C., Souza, D. D., Kimpton, A., & Ljbusic, J. (2021). Exploring higher education students’ critical thinking skills through content analysis. Thinking Skills and Creativity, 41(100877), 1–9. https://doi.org/10.1016/j.tsc.2021.100877

Hattie, J., & Gan, M. (2011). Instruction based on feedback. Handbook of research on learning and instruction.

Kholid, M. N., Hamida, P. S., Pradana, L. N., & Maharani, S. (2020). Students’ Critical Thinking Depends On Their Cognitive Style. International Journal of Scientific & Technology Research, 9(01), 1045–1049.

Khonamri, F., & Karimabadi, M. (2015). Collaborative Strategic Reading and Critical Reading Ability of Intermediate Iranian Learners. Theory and Practice in Language Studies, 5(7), 1375–1382. http://dx.doi.org/10.17507/tpls.0507.09

Kong, L.-N., Qin, B., Zhou, Y., Mou, S., & Gao, H.-M. (2014). The effectiveness of problem-based learning on development of nursing students’ critical thinking: A systematic review and meta-analysis. International Journal of Nursing Studies, 51, 458–469. https://doi.org/10.1016/j.ijnurstu.2013.06.009

Ku, K. Y. L., Kong, Q., Song, Y., Deng, L., Kang, Y., & Hu, A. (2019). What predicts adolescents’ critical thinking about real-life news? The roles of social media news consumption and news media literacy. Thinking Skills and Creativity, 33(100570), 1–12. https://doi.org/10.1016/j.tsc.2019.05.004

Kuhn, D. (2019). Critical Thinking as Discourse. Human Development, 62(3), 146–164. https://doi.org/10.1159/000500171

Lai, E. R. (2011). Critical Thinking: A Literature Review. Pearson’s Research Reports, 6(1), 1–49. https://doi.org/10.1046/j.1537-2995.1995.35395184278.x

Lin, S. (2013). Science and non-science undergraduate students’ critical thinking and argumentation performance in reading a science news report. International Journal of Science and Mathematics Education, 12(5), 1023–1046. https://doi.org/10.1007/s10763-013-9451-7

Liu, O. L., Frankel, L., & Roohr, K. C. (2014). Assessing Critical Thinking in Higher Education: Current State and Directions for Next-Generation Assessment. ETS Research Report Series, 1, 1–23.

Pearl, A. O., Rayner, G. M., Larson, I., & Orlando, L. (2018). Thinking about critical thinking: An industry perspective. Industry and Higher Education, 1–11. https://doi.org/10.1177/0950422218796099

Pui, P., Yuen, B., & Goh, H. (2020). Using a Criterion-referenced Rubric to Enhance Student Learning: a case study in a critical thinking and writing module. Higher Education Research & Development. https://doi.org/10.1080/07294360.2020.1795811

Reynders, G., Lantz, J., Ruder, S. M., Stanford, C. L., & Cole, R. S. (2020). Rubrics to assess critical thinking and information processing in undergraduate STEM courses. International Journal of STEM Education Https://Doi.Org/10.1186/S40594-020-00208-5, 7(9), 1–15. https://doi.org/10.1186/s40594-020-00208-5

Saxton, E., Belanger, S., & Becker, W. (2012). The Critical Thinking Analytic Rubric (CTAR): Investigating intra-rater and inter-rater reliability of a scoring mechanism for critical thinking performance assessments. Assessing Writing, 17, 251–270.

Shaw, A., Liu, O. L., Gu, L., Kardonova, E., Chirikov, I., Li, G., Hu, S., Yu, N., Ma, L., Guo, F., Su, Q., Shi, J., & Loyalka, P. (2019). Thinking critically about critical thinking: validating the Russian HEIghten ® critical thinking assessment. Studies in Higher Education, 2–16. https://doi.org/10.1080/03075079.2019.1672640

Smit, R., & Birri, T. (2014). Assuring the quality of standards-oriented classroom assessment with rubrics for complex competencies. Studies in Educational Evaluation, 1–9. https://doi.org/10.1016/j.stueduc.2014.02.002

Stein, B., & Haynes, A. (2011). Engaging Faculty in the Assessment and Improvement of Students’ Critical Thinking Using the Critical Thinking Assessment Test By Barry Stein and Ada Haynes. Change: The Magazine of Higher Learning, 43(2), 44–49.

Tong, F., Tang, S., Irby, B. J., Lara-alecio, R., & Guerrero, C. (2020). The determination of appropriate coe ffi cient indices for inter-rater reliability : Using classroom observation instruments as fi delity measures in large-scale randomized research. International Journal of Educational Research, 99(101514), 1–10. https://doi.org/10.1016/j.ijer.2019.101514

Undergraduate Professional Education in Chemistry ACS Guidelines and Evaluation Procedures for Bachelor’s Degree Programs. (2015). American Chemical Society Committee on Professional Training.

VALUE Rubric Development Project. (2019). Association of American Colleges and Universities. https://www.aacu.org/value/rubrics

Vision for science and mathematics education. (2014). The Royal Society Science Policy Centre.

Wale, B. D., & Bishaw, K. S. (2020). Effects of using inquiry-based learning on EFL students’ critical thinking skills. Asian-Pacific Journal of Second and Foreign Language Education, 5(9), 1–14.

Downloads

Published

2024-03-09

How to Cite

Hidayati, N., Suryanti, S., Rahmayumita, R., & Aisya, S. (2024). Development of Critical Thinking Skills Instruments : Cases for Essay Tests. Jurnal Kependidikan: Jurnal Hasil Penelitian Dan Kajian Kepustakaan Di Bidang Pendidikan, Pengajaran Dan Pembelajaran, 10(1), 77–88. https://doi.org/10.33394/jk.v10i1.10052

Issue

Vol. 10 No. 1 (2024): March

Section

Articles

Citation Check

License

License and Publishing Agreement
In submitting the manuscript to the journal, the authors certify that:
  • They are authorized by their co-authors to enter into these arrangements.
  • The work described has not been formally published before, except in the form of an abstract or as part of a published lecture, review, thesis, or overlay journal.
  • That it is not under consideration for publication elsewhere,
  • That its publication has been approved by all the author(s) and by the responsible authorities – tacitly or explicitly – of the institutes where the work has been carried out.
  • They secure the right to reproduce any material that has already been published or copyrighted elsewhere.
  • They agree to the following license and publishing agreement.
Copyright
Authors who publish with JK agree to the following terms:
  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (CC BY-SA 4.0) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal. 
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
Licensing for Data Publication
JK use a variety of waivers and licenses, that are specifically designed for and appropriate for the treatment of data:
Other data publishing licenses may be allowed as exceptions (subject to approval by the editor on a case-by-case basis) and should be justified with a written statement from the author, which will be published with the article.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.