Paper vs. Digital Assessments: Evaluating Critical Thinking on Ecological Issues in Indonesian Madrasahs

Keywords: Assessment, Critical Thinking Skills, Elementary Education, Madrasah Ibtidaiyah

Viewed = 0 time(s)

Abstract

This research aims to determine the difference in the assessment results in the critical thinking skills of State Madrasah Ibtidaiyah (MIN) students or State Islamic schools in Indonesia using paper-based and digital-based tests. Data was collected from 314 participants representing MIN on four large islands in Indonesia, namely MIN Bogor Regency, West Java, MIN Sinjai Regency, South Sulawesi, MIN Palembang City, South Sumatra, and MIN Balikpapan City, East Kalimantan. The research method used a quantitative survey. A total of 24 valid Critical Thinking Skills Items with the Facione Critical Thinking Skills indicator as an instrument for measuring critical thinking skills. Factors such as motivation, technological accessibility, and effectiveness of test formats contribute to differences in results. This study provides new insights into the effectiveness of technology-based assessment in Madrasa education, especially in evaluating critical thinking skills. These findings lead to policy recommendations for improving the Madrasa assessment system in the digital era.



Downloads

Download data is not yet available.

References

Almeida, B., Santos, M., & Justi, R. (2023). Aspects and abilities of science literacy in the context of nature of science teaching. Science \& Education, 32(3), 567–587. https://doi.org/10.1007/s11191-022-00324-4
Anggraeni, D. M., Prahani, B. K., Suprapto, N., Shofiyah, N., & Jatmiko, B. (2023). A systematic review of problem-based learning research in fostering critical thinking skills. Thinking Skills and Creativity, 49, 101334. https://doi.org/https://doi.org/10.1016/j.tsc.2023.101334
Arenas, M. R., & Palaoag, T. D. (2018). Gamified learning system: A motivation for exceptional learners of SPED-baguio for better engagement. Journal of Advanced Research in Dynamical and Control Systems, 10(11), 216 – 222.https://www.scopus.com/inward/record.uri?eid=2-s2.0-85074669686&partnerID=40&md5=ec4f0e976f1bef2043b4ca194a43a4f2
Ashari, Z. M., & Sun, D. (2024). Project Based Learning on Promoting Children’s Critical Thinking Skills: A Systematic Review. International Journal of Academic Research in Progressive Education and Development. https://doi.org/10.6007/ijarped/v13-i3/21732
Azra, A. (2021). Understanding Indonesia's Third Way of Islam. In Wars of Ideas: Theology, Interpretation and Power in the Muslim World (p. 73). Rowman & Littlefield.
Bacak, J., Martin, F., Ahlgrim-Delzell, L., Polly, D., & Wang, W. (2022). Elementary Educator Perceptions of Student Digital Safety Based on Technology Use in the Classroom. Computers in the Schools, 39, 186–202. https://doi.org/10.1080/07380569.2022.2071233
Bainiashova, S. (2023). Content analysis of the concept of" information literacy". Zhytomyr Ivan Franko State University Journal. Рedagogical Sciences, 3 (114), 5–15. https://doi.org/ 10.35433/pedagogy.3(114).2023.5-15
Baviskar, S. N., Hartle, R., & Whitney, T. (2009). Essential Criteria to Characterize Constructivist Teaching: Derived from a literature review and applied to five constructivist‐teaching method articles. International Journal of Science Education, 31, 541–550. https://doi.org/10.1080/09500690701731121
Brüggemann, T., Ludewig, U., Lorenz, R., & McElvany, N. (2023a). Effects of mode and medium in reading comprehension tests on cognitive load. Computers & Education, 192, 104649. https://doi.org/10.1016/j.compedu.2022.104649
Brüggemann, T., Ludewig, U., Lorenz, R., & McElvany, N. (2023b). Effects of test mode and medium on elementary school students’ test experience. European Journal of Psychological Assessment. https://doi.org/10.1027/1015-5759/a000767
Carpenter, R., & Alloway, T. (2019). Computer Versus Paper-Based Testing: Are They Equivalent When it Comes to Working Memory? Journal of Psychoeducational Assessment, 37, 382–394. https://doi.org/10.1177/0734282918761496
Christensen, E. W., & Bailey, J. R. (1997). A source accessibility effect on media selection. Management Communication Quarterly, 10(3), 373–387. https://doi.org/10.1177/0893318997010003005
Ciampa, K., & Gallagher, T. (2013). Getting in Touch: Use of Mobile Devices in the Elementary Classroom. Computers in the Schools, 30, 309–328. https://doi.org/10.1080/07380569.2013.846716
Clariana, R. B., & Smith, L. (1988). Learning Style Shifts in Computer-Assisted Instructional Settings. The Annual Meeting of the International Association for Computers in Education Held in Conjunction with the American Educational Research Association, 20. https://eric.ed.gov/?id=ED295796
Dewi, N. K. L., Utami, I. G. A. L. P., & Santosa, M. H. (2022). Development of Digital Supplementary Material Using Quizizz-Based Learning Media in Intensive English Course. Jurnal Inovasi Dan Teknologi Pembelajaran (JINOTEP): Kajian Dan Riset Dalam Teknologi Pembelajaran, 9(1), 14–22. https://doi.org/10.17977/um031v9i12022p014
Ennis, R. H. (1989). Critical thinking and subject specificity: Clarification and needed research. Educational Researcher, 18(3), 4–10. https://doi.org/10.3102/0013189X018003004
Erwin, T. D. (2000). The NPEC Sourcebook on Assessment, Volume 1: Definitions and Assessment Methods for Critical Thinking, Problem Solving, and Writing. https://nces.ed.gov/pubs2000/2000195.pdf#page=20.11
Facione, P. (1990). Critical thinking: A statement of expert consensus for educational assessment and instruction (The Delphi Report). https://philarchive.org/archive/faccta
Facione, P. A., & others. (2011). Critical thinking: What it is and why it counts. Insight Assessment, 1(1), 1–23.
Fajari, L. E. W., & others. (2021). Open-Ended Questions to Assess Critical-Thinking Skills in Indonesian Elementary School. International Journal of Instruction, 14(1), 615–630. https://doi.org/10.29333/iji.2021.14137a
Ferrary, C. H., Kawuryan, S. P., & others. (2023). How are Critical Thinking Skills and Integration of IPS Problems in Elementary Schools? Mimbar Ilmu, 28(1), 147–155. https://doi.org/10.23887/mi.v28i1.55641
Fitanti, S. Y., Prahani, B. K., & Suryanti, S. (2024). Profile of Critical Thinking Skills of Grade IV Elementary School Students on Energy Source Material. Scaffolding: Jurnal Pendidikan Islam Dan Multikulturalisme, 6(3), 58–74. https://doi.org/10.37680/scaffolding.v6i3.6284
Gelerstein, D., Del Rio, R., Nussbaum, M., Chiuminatto, P., & López, X. (2016). Designing and implementing a test for measuring critical thinking in primary school. Thinking Skills and Creativity, 20, 40–49. https://doi.org/10.1016/j.tsc.2016.02.002
Gendenjamts, S. (2023). Measuring Higher-Order Thinking Skills in Science Among Primary School Students Using Item Response Theory. European Journal of Education Studies, 10(12). http://dx.doi.org/10.46827/ejes.v10i12.5089
Göksün, D. O., & Gürsoy, G. (2019). Comparing success and engagement in gamified learning experiences via Kahoot and Quizizz. Computers & Education, 135, 15–29. https://doi.org/10.1016/j.compedu.2019.02.015
Guo, R., Jantharajit, N., & Thongpanit, P. (2024). Construct an instructional approach based on collaborative and reflective learning to enhance students' analytical and critical thinking skills. Asian Journal of Contemporary Education. https://doi.org/10.55493/5052.v8i2.5184
Halkiopoulos, C., & Gkintoni, E. (2024). Leveraging AI in e-learning: Personalized learning and adaptive assessment through cognitive neuropsychology—A systematic analysis. Electronics, 13(18), 3762. https://doi.org/10.3390/electronics13183762
Halpern, D. F. (2013). Thought and knowledge: An introduction to critical thinking (5th ed.). Psychology press. https://doi.org/10.4324/9781315885278
Hasanah, S. N., Sunarno, W., & Prayitno, B. A. (2021). Improving students’ critical thinking skills trough contextual teaching and learning science module. JIPF (Jurnal Ilmu Pendidikan Fisika), 6(2), 106. https://dx.doi.org/10.26737/jipf.v6i2.1828
Hitchcock, D. (2018). Critical thinking. Stanford Encyclopedia of Philosophy Archive. https://stanford.library.sydney.edu.au/archives/spr2019/entries/critical-thinking/
Hollett, E. C. (2022). Analytical Thinking for Advances Learners, Grades 3-5. ROUTLEDGE. https://doi.org/10.4324/9781003268338
Hsu, F.-H., Lin, I.-H., Yeh, H.-C., & Chen, N.-S. (2022). Effect of Socratic Reflection Prompts via video-based learning system on elementary school students’ critical thinking skills. Computers & Education, 183, 104497. https://doi.org/https://doi.org/10.1016/j.compedu.2022.104497
Hussein, M. H., Ow, S. H., Cheong, L. S., & Thong, M. (2019). A Digital Game-Based Learning Method to Improve Students’ Critical Thinking Skills in Elementary Science. IEEE Access, 7, 96309–96318. https://doi.org/10.1109/ACCESS.2019.2929089
Ishartono, N., Desstya, A., Prayitno, H., & Sidiq, Y. (2021). The Quality of HOTS-Based Science Questions Developed by Indonesian Elementary School Teachers. Journal of Education Technology. https://doi.org/10.23887/jet.v5i2.33813
Isrokatun, I., Hanifah, N., Rohman, Y. A., Rosmiati, R., & Khoerunnisah, R. (2022). Enhancing critical thinking skills for low-grade elementary school students using mobile apps. Cypriot Journal of Educational Sciences, 17(9), 3218 – 3237. https://doi.org/10.18844/cjes.v17i9.7422
Jeong, S., Sherman, B., & Tippins, D. J. (2021). The Anthropocene as we know it: Posthumanism, science education and scientific literacy as a path to sustainability. Cultural Studies of Science Education, 16, 805–820. https://doi.org/10.1007/s11422-021-10029-9
Jonassen, D. H. (2013). Evaluating constructivistic learning. In Constructivism and the technology of instruction (pp. 137–148). Routledge. https://doi.org/10.4324/9780203461976
Juliyantika, T., & Batubara, H. H. (2022). Tren Penelitian Keterampilan Berpikir Kritis pada Jurnal Pendidikan Dasar di Indonesia. Jurnal Basicedu, 6(3), 4731–4744. https://doi.org/10.31004/basicedu.v6i3.2869
Kannengiesser, U., & Gero, J. S. (2019). Design thinking, fast and slow: a framework for Kahneman’s dual-system theory in design. Design Science, 5. https://doi.org/10.1017/dsj.2019.9
Kawuryan, S. P., Sayuti, S., & Aman, A. (2022). A Descriptive Study of Critical Thinking Abilities of Elementary School Students. Jurnal Cakrawala Pendidikan. https://doi.org/10.21831/cp.v41i1.44322
Kennedy, M., Fisher, M. B., & Ennis, R. H. (1991). Critical thinking: Literature review and needed research. Educational Values and Cognitive Instruction: Implications for Reform, 2, 11–40. https://doi.org/10.4324/9781315044392
Kilickaya, F., & Ersoy, M. A. (2016). Teaching pre-service language teachers how to write instructional objectives through the ABCD Model. New Insights into Language Teaching and Learning Practices, 41–52. https://eric.ed.gov/?id=ED571537
Kisilowska-Szurmińska, M. (2023). Information Overload as a Burden and a Challenge. What Can We Learn for Information Literacy? European Conference on Information Literacy, 161–172. https://doi.org/10.1007/978-3-031-52998-6_14
Ku, K. Y. (2009). Assessing Students’ Critical Thinking Performance: Urging for Measurements Using Multi-Response Format. Thinking Skills and Creativity, 4, 70–76. https://doi.org/10.1016/J.TSC.2009.02.001
Kurniawan, D. A., Wirayuda, R. P., Putri, W. A., Rini, E. F. S., Ginting, A. A. B., Ratnawati, T., & others. (2023). Impact of science process skills on thinking skills in rural and urban schools. International Journal of Instruction, 16(2), 803–822. https://www.e-iji.net/dosyalar/iji_2023_2_42.pdf
Kurniawati, & Istiyono, E. (2022). Developing a Critical Thinking Test in Social Science for Grade V Elementary School. Advances in Social Science, Education and Humanities Research. https://doi.org/10.2991/assehr.k.220129.040
Kusnanto, N., Sukristyanto, A., & Rochim, A. I. (2023). Relevance Of National Education Policies As An Effort To Improve The Quality Of Madrasah Tsanawiyah Education Services. THE SPIRIT OF SOCIETY JOURNAL: International Journal of Society Development and Engagement, 6(2), 136–151. https://doi.org/10.29138/scj.v6i2.2210
Lee, S.-A., Hong, J.-H., Kim, N.-Y., Min, H.-M., Yang, H.-M., Lee, S.-H., Choi, S.-J., & Park, J.-H. (2024). Unveiling Neurocognitive Disparities in Encoding and Retrieval between Paper and Digital Tablet-Based Learning. Brain Sciences, 14(1), 76. https://doi.org/10.3390/brainsci14010076
Lin, I.-H., Chew, S., & Chen, N. (2017). A Vocal Assessment Approach to Measure Elementary School Students’ Critical Thinking Skills. 2017 IEEE 17th International Conference on Advanced Learning Technologies (ICALT), 419–421. https://doi.org/10.1109/ICALT.2017.30
Manurung, A. S., & Pappachan, P. (2025). The role of discovery learning in efforts to develop students’ critical thinking abilities. Journal of Education and Learning, 19(1), 46 – 53. https://doi.org/10.11591/edulearn.v19i1.21788
Maraza-Quispe, B. (2024). Impact of the Use of Gamified Online Tools: A Study with Kahoot and Quizizz in the Educational Context. International Journal of Information and Education Technology. https://doi.org/10.18178/ijiet.2024.14.1.2033
Mariana, D., & Helmi, A. M. (2022). Madrasah Sebagai Lembaga Pendidikan di Indonesia. Jurnal Pendidikan Tambusai, 6(1), 1907–1919. https://jptam.org/index.php/jptam/article/download/3236/2699/6188
Martani, E. (2023). Improving the quality of madrasah education through madrasah resource management. Dawuh Guru: Jurnal Pendidikan MI/SD, 3(1), 25–38. https://doi.org/10.35878/guru.v3i1.481
Mason, M. (2007). Critical thinking and learning. Educational Philosophy and Theory, 39(4), 339–349. https://doi.org/10.1111/j.1469-5812.2007.00343.x
Mastuti, A. G., Abdillah, A., Sehuwaky, N., & Risahondua, R. (2022). Revealing students’ critical thinking ability according to facione’s theory. Al-Jabar: Jurnal Pendidikan Matematika, 13(2), 261–272. file:///C:/Users/HP/Downloads/13005-47003-1-PB.pdf
Melawati, Y., Rosidin, U., Nurhanurawati, N., & Handoko, H. (2023). Development of A Higher-Order Thinking Skills (HOTS) Instrument to Measure Student’s Critical Thinking Abilities in Class V Elementary School. International Journal of Current Science Research and Review. https://doi.org/10.47191/ijcsrr/v6-i12-70
Millah, M. F., Jaenudin, R., Nurhayati, E., Mulyani, S., Nurmaulidah, E. F., & Nasrudin, D. (2022). AKMI: Initial Knowledge, misconceptions, and readiness of madrasa teachers. International Conference on Madrasah Reform 2021 (ICMR 2021), 154–159. https://doi.org/10.2991/assehr.k.220104.023
Nardi, A., & Ranieri, M. (2019). Comparing paper-based and electronic multiple-choice examinations with personal devices: Impact on students’ performance, self-efficacy and satisfaction. British Journal of Educational Technology, 50(3), 1495–1506. https://doi.org/10.1111/bjet.12644
Owan, V. J., Abang, K. B., Idika, D. O., Etta, E. O., & Bassey, B. A. (2023). Exploring the potential of artificial intelligence tools in educational measurement and assessment. Eurasia Journal of Mathematics, Science and Technology Education, 19(8), em2307. https://doi.org/10.29333/ejmste/13428
Park, J., & Niyozov, S. (2008). Madrasa education in South Asia and Southeast Asia: current issues and debates. Asia Pacific Journal of Education, 28(4), 323–351. https://doi.org/10.1080/02188790802475372
Perry, K., Meissel, K., & Hill, M. F. (2022). Rebooting assessment. Exploring the challenges and benefits of shifting from pen-and-paper to computer in summative assessment. Educational Research Review, 36, 100451. https://doi.org/https://doi.org/10.1016/j.edurev.2022.100451
Primasari, R., Miarsyah, M., & Rusdi, R. (2020). Science literacy, critical thinking skill, and motivation: A correlational study. JPBI (Jurnal Pendidikan Biologi Indonesia), 6(2), 273–282.
Queiruga-Dios, M. Á., López-Iñesta, E., Diez-Ojeda, M., Sáiz-Manzanares, M. C., & Vázquez Dorr\'\io, J. B. (2020). Citizen science for scientific literacy and attaining sustainable development goals in formal education. Sustainability, 12(10), 4283. https://doi.org/10.22219/jpbi.v6i2.11124
Ridzal, D. A., & Haswan, H. (2023). Analysis of the correlation between science literacy and critical thinking of grade eight students in the circulatory system. Jurnal Pijar Mipa, 18(1), 1–5. https://doi.org/ 10.29303/jpm.v18i1.4469
Rintayati, P., Lukitasari, H., & Syawaludin, A. (2021). Development of Two-Tier Multiple Choice Test to Assess Indonesian Elementary Students’ Higher-Order Thinking Skills. International Journal of Instruction. https://doi.org/10.29333/iji.2021.14133a
Rosa, N. (2022). Selain ANBK, Madrasah juga Wajib Ikut AKMI, Apa Itu? Detikedu. https://www.detik.com/edu/sekolah/d-6289196/selain-anbk-madrasah-juga-wajib-ikut-akmi-apa-itu
Sabil, H., Asrial, Syahrial, Kurniawan, D. A., Perdana, R., Nawahdani, A. M., Widodi, B., & Belajar, P. M. (2024). Students’ Learning Interest Using Web-Based Assessment at The Elementary School Level. Jurnal Ilmiah Sekolah Dasar. https://doi.org/10.23887/jisd.v7i4.59737
Sarwanto, S., Fajari, L. E. W., & Chumdari, C. (2021). Critical thinking skills and their impacts on elementary school students. Malaysian Journal of Learning and Instruction, 18(2), 161. https://doi.org/10.32890/mjli2021.18.2.6
Shin-Jia, H., & Tien-Chi, H. (2024). Integrating the United Nations’ sustainable development goals into STEAM education: A problem-based learning approach in elementary and secondary education. Journal of Textbook Research, 17(2), 79. https://doi.org/ 10.6481/JTR.202408_17(2).03
Shute, V., & Rahimi, S. (2017). Review of computer-based assessment for learning in elementary and secondary education. J. Comput. Assist. Learn., 33, 1–19. https://doi.org/10.1111/jcal.12172
Shyr, W., Hsieh, Y., & Chen, C.-H. (2021). The Effects of Peer-Based Instant Response System to Promote Learning Performance, Intrinsic Motivation and Self-Efficacy. Sustainability. https://doi.org/10.3390/SU13084320
Sofanudin, A., Rokhman, F., & others. (2016). Quality-Oriented Management of Educational Innovation at Madrasah Ibtidaiyah. Journal of Education and Practice, 7(27), 176–180. https://eric.ed.gov/?id=EJ1115847
Sumarni, W., & Kadarwati, S. (2020). Ethno-stem project-based learning: Its impact to critical and creative thinking skills. Jurnal Pendidikan IPA Indonesia, 9(1), 11–21. https://doi.org/10.15294/jpii.v9i1.21754
Sutiani, A., Situmorang, M., & Silalahi, A. (2021). Implementation of an inquiry learning model with science literacy to improve student critical thinking skills. International Journal of Instruction, 14(2), 117–138. https://digilib.unimed.ac.id/id/eprint/42164/3/ARTICLE.pdf
Swiecki, Z., Khosravi, H., Chen, G., Martinez-Maldonado, R., Lodge, J. M., Milligan, S., Selwyn, N., & Gašević, D. (2022). Assessment in the age of artificial intelligence. Computers and Education: Artificial Intelligence, 3, 100075. https://doi.org/10.1016/j.caeai.2022.100075
Tanti, T., Kurniawan, D. A., Kuswanto, K., Utami, W., & Wardhana, I. (2020). Science process skills and critical thinking in science: Urban and rural disparity. Jurnal Pendidikan IPA Indonesia, 9(4), 489–498. https://doi.org/ 10.15294/jpii.v9i4.24139
Ubaidillah, M., Hartono, H., Marwoto, P., Wiyanto, W., & Subali, B. (2023). How to Improve Critical Thinking in Physics Learning? A Systematic Literature Review. Journal of Educational, Cultural and Psychological Studies (ECPS Journal), 28, 161–187. https://doi.org/10.7358/ecps-2023-028-ubai
Utaminingsih, S., Amalia, I., & Sumaji, S. (2024). Management of Mathematics Learning Based on Interactive Digital Worksheets to Improve Students’ Critical Thinking Ability. Journal of Curriculum and Teaching. https://doi.org/10.5430/jct.v13n1p159
Vieira, R., & Tenreiro-Vieira, C. (2016). Fostering Scientific Literacy and Critical Thinking in Elementary Science Education. International Journal of Science and Mathematics Education, 14, 659–680. https://doi.org/10.1007/S10763-014-9605-2
Wani, S. A., & Hussian, Z. (2024). Developing Critical Thinking Skills: Encouraging Analytical and Creative Thinking. In Transforming Education for Personalized Learning (pp. 114–130). IGI Global. https://doi.org/ 10.4018/979-8-3693-0868-4.ch007
Widhiarso, W., & Ridho, A. (2022). Validation of Setting and Design of Multi-Stage Testing (MST) to Portray Students’ Achievement on Reading Literacy in AKMI 2021. International Conference on Madrasah Reform 2021 (ICMR 2021), 6–10. https://doi.org/10.2991/assehr.k.220104.002
WorldBank. (2024). Realizing Education’s Promise: Support to Indonesia’s Ministry of Religious Affairs for Improved Quality of Education. https://projects.worldbank.org/en/projects-operations/project-detail/P168076
Xu, Ling and Abdullah, Tina and Liu, Qian and ‘Ain Mohd Shahroom, Nur, Effects of Critical Thinking Disposition on Foreign Language Proficiency in Foreign Language Learning: Evidence from China (February 7, 2023). Arab World English Journal (AWEJ) Special Issue on Communication and Language in Virtual Spaces, January 2023 , Available at SSRN: https://ssrn.com/abstract=4350357 or http://dx.doi.org/10.2139/ssrn.4350357
Yousefizadeh, S. (2023). Evaluation of Curriculum Programs Based on Constructivist Learning Theory and the Development of Critical Thinking. Journal of Study and Innovation in Education and Development. https://doi.org/10.61838/jsied.3.1.4
Zarkasy, A., Chotimah, C., & Mahzuna, L. (2023). Implementing Massive Online Open Course (MOOC) through "Pintar" In Improving Teacher Competence. Annual International Conference on Islamic Education for Students, 2(1), 665–674. https://doi.org/10.18326/aicoies.v2i1.620
Published
2025-04-30
Section
Articles
How to Cite
Berlian, Z., Isnaini, M., Farwati , R., Metafisika, K., Velina, Y., & Fitriyanni, V. (2025). Paper vs. Digital Assessments: Evaluating Critical Thinking on Ecological Issues in Indonesian Madrasahs. Journal of Applied Science, Engineering, Technology, and Education, 7(1), 41-55. https://doi.org/10.35877/454RI.asci3785