Journal of Education for Sustainability and Transformation (JESTRA)

Peer-reviewed open-access journal

ISSN: XXXX-XXXX

Vol. 1 No. 1 (2025)
Articles

Development of a Four-Tier Test (FTT) in the Form of a Virtualization Static Fluid Test (VSFT) Using Rasch Model Analysis to Support Learning During the COVID-19 Pandemic

  • Neri Anggraini,
  • Bambang Heru Iswanto,
  • Firmanul Catur Wibowo

Keywords

  • Four Tier Test
  • Virtualization Static Fluid Test
  • Rasch Model

Abstract

This study aims to develop a valid and reliable test instrument. The instrument is a Four-Tier Test (FTT) in the form of a Virtual Static Fluid Test (VSFT) on Static Fluids material, analyzed using the Rasch Model. The research method employed was research and development (R&D) with the 4D model. The study produced an educational product in the form of an FTT-based VSFT, consisting of 20 test items. Expert validation results yielded an average score of 88.13% with a “very good” category. The CVR value of each item was 1, while the CVI value was also 1, both categorized as “very good.” Practitioner validation results obtained an average score of 89.58% with a “very good” category. Rasch Model analysis showed that the test instrument was acceptable and capable of measuring what it was intended to measure. The reliability of all items was categorized as good, all items were valid, and they were distributed across three levels of difficulty. Furthermore, no item showed bias related to gender differences. The results of students’ misconception analysis indicated that only 4.68% fully understood the concepts, 23.42% partially understood, 68.95% experienced misconceptions, and 12.95% did not understand the concepts at all. The conclusion of this study is that the FTT-based VSFT instrument is valid and reliable as an assessment tool during the COVID-19 pandemic. In addition, the FTT-based VSFT instrument is also effective in analyzing students’ conceptions.

PDF

References

  1. Abbas, M. L. (2020). Pengembangan Computer Based Diagnostic Test Misconception Mahasiswa pada Materi Suhu dan Kalor. Jurnal Pendidikan Fisika dan Keilmuan, 6(1), 12-24.
  2. Akbar, M. N., Firman, H., & Rusyati, L. (2017). Developing Science Virtual Test to Measure Students' Critical Thinking in Living Things and Environmental Sustaibility Theme. Journal of Physics.
  3. Al-Taweel, D., & Awad, A. (2020). Development and validation of medication assessment tools to evaluate prescribing adherence to evidence-based guidelines for secondary prevention of coronary heart disease in post-acute coronary syndromes patients in Kuwait. PLoS One, 15(11), 1-16.
  4. Alwi, I. (2017). SENSITIVITY OF MANTEL HAENSZEL MODEL AND RASCH MODEL AS VIEWED FROM SAMPLE SIZE. Jurnal Evaluasi Pendidikan, 2(1), 18-29.
  5. Al-Zoubi, F., Mayo, N., Rochette, A., & Thomas, A. (2018). Applying modern measurement approaches to constructs relevant to evidence-based practice among Canadian physical and occupational therapists. Implementation Science, 13, 1-21.
  6. Amin, N. W., & Samsudin, A. (2016). Analisis Instrumen Tes Diagnostik Dynamic-Fluid Conceptual CHange Inventory (DFCCI) Bentuk Four Tier Test pada Beberapa SMA di BAndung Raya. Simposium Nasional Inovasi dan Pembelajaran Sains (SNIPS), 570 574.
  7. Anderson, R. M., Heesterbeek, H., Klinkenberg, D., & Hollingsworth, T. (2020). How will country-based mitigation measures influence the course of the COVID-19 epidemic? Lancet, 931-934.
  8. Anselmi, P., Colledani, D., & Robusto, E. (2019). A Comparison of Classical and Modern Measures of Internal Consistency. Frontier in Psychology, 10, 1-12.
  9. Arikunto, S. (2010). Prosedur Penelitian Suatu Pendekatan Praktik (Edisi Revisi 2010). Jakarta: Rineka Cipta. Arikunto, S. (2012). Dasar-dasar Evaluasi Pendidikan Edisi 2. Jakarta: Bumi Aksara.
  10. Aslanoglu, A. E., Karakaya, I., & Sata, M. (2020). Evaluation of University Students' Rating Behaviors in Self and Peer Rating Process via Many Facet Rasch Model. Eurasian Journal of Educational Research, 89, 25-46.
  11. Ayre, C., & Scally, A. J. (2014). Critical values for Lawshe's content validity ratio: Revisiting the original methods of calculation. Measurement and Evaluation in Counseling and Development. 47(1), 79-86. https://doi.org/10.1177/0748175613513808
  12. Azizah, & Wahyuningsih, S. (2020). PENGGUNAAN MODEL RASCH UNTUK ANALISIS INSTRUMEN TES PADA MATA KULIAH MATEMATIKA AKTUARIA. Jurnal Pendidikan Matematika, 3(1), 45-50.
  13. Barus, G., Sinaga, J. D., & Moron, Y. D. (2019). Testing the quality of film-based creative and innovative character education test with rasch model. Jurnal Konseling dan Pendidikan, 7(2), 71-80.
  14. Bichi, A. A., Talib, R., Atan, N. A., Ibrahim, H., & Yusof, S. M. (2019). Validation of a developed university placement test using classical test theory and Rasch measurement approach. International Journal of Advanced and Applied Sciences, 6(6), 22-29.
  15. Bodzin, A., Hammond, T., Fu, Q., & Farina, W. (2020). Development of Instruments to Assess Students' Spatial Learning Attitudes (SLA) and Interest in Science, Technology and Geospatial Technology (STEM-GEO). International Journal of Educational Methodology, 6(1), 67-81.
  16. Boone, W. J. (2016). Rasch Analysis for Instrument Development: Why, When, and How? CBE Life Sciences Education, 15(4).
  17. Boone, W. J., & Yale, M. S. (2014). Rasch Analysis in the Human Science. Springer.
  18. Borkan, B. (2017). Exploring Variability Sources in Student Evaluation of Teaching via Many Facet Rasch Model. Journal of Measurement and Evaluation in Education and Psychology, 8(1), 15-33.
  19. BPS, B. P. (2019). Potret Pendidikan Indonesia; Statistik Pendidikan Indonesia 2019. Badan Pusat Statistik. Jakarta: Badan Pusat Statistik.
  20. Buth, L., Juraschek, M., Sangwan, K. S., Herrmann, C., & Thiede, S. (2020). Integrating virtual and physical production proccess in learning factories. Procedia Manufacturing, 121 127.
  21. Celeon, I., & R, S. (2010). Development and application of a threetier diagnostic test to assesss secondary student's understanding of waves. International Journal of Science Education, 32(7), 939-961.
  22. Cetin, A., Dindar, & Geban, O. (2011). Development of a three-tier test to assess high school students' understanding of acids and bases. Procedia Social and Behavior Science, 15, 600-604.
  23. Chang, J., Li, Y., Song, H., Yong, L., Luo, L., Zhang, Z., & Song, N. (2020). Assessment of Validity of Children's Movement Skill Quotient (CMSQ) Based on the Physical Education Classroom Environment. BioMed Research International, 1-11.
  24. Clariana, R., & Wallace, P. (2002). Learning and Assessment I. British Journal of Educational Technology, 593-602.
  25. Daryanto. (2012). Evaluasi Pendidikan. Jakarta: PT Rineka Cipta. Dhitareka, P. H., Firman, H., & Rusyati, L. (2018). The comparison between science virtual and paper best test in measuring grade 7 students' critical thingking. Journal of Physics; Conf. Series.
  26. Fariyani, Q. (2015). Pengembangan Four-tier Diagnostic Test untuk Mengungkap Miskonsepsi Fisika SMA Kelas X. Journal of Innovation Science Education, 4.
  27. Fenditasari, K. (2020). Identification of misconceptions on heat and temperature among physics education students using four-tier diagnostic test. Jurnal of Physics: Conference Series, 1-12.
  28. Feng, Guobao et al. (2020). Electron beam irradiation on novel coronavirus (COVID-19): A Monte-Carlo simulation. Chin. Phys. B, 29(04).
  29. Giancoli, D. C. (1999). Fisika Dasar Edisi Kelima Jilid 1; alih bahasa. Jakarta: Erlangga.
  30. Gugus Tugas Percepatan Penangan Covid-19. (2020). Panduan Penyelenggaraan dan Pembelajaran pada Tahun Ajaran dan Tahun Akademik Baru di Masa Pandemi Covid 19. Kemendikbud; Kemenag; Kemenkes; Kemendagri.
  31. Fuadi, F. N., Yulianto, A., & Akhirani, Y. S. (2020). Items Quality Analysis Using Rasch Model To Measure Elementary School Students' Critical Thinking Skill On Stem Learning. Jurnal Pendidikan Indonesia, 9(1), 61-74.
  32. Hermita, N., Suhandi, A., Syaodih, E., Samsudin, A., Isjoni, Johan, H., . . . Safitri, D. (2017). Constructing and Implementing a Four Tier Test about Static Electricity to Diagnose Pre-service Elementary School Teacher' Misconception. International Conference in Mathematics and Sciences Education (ICMScE).
  33. Hestenes, M. W., & Swackhamer. (1992). Force Concept Inventory. The Physics Teacher, 30(3), 141-158.
  34. Hunter, P. (2015). The virtual university: digital tools for e-learning and remote learning are becoming an increasingly important tool for teaching at universitas. EMBO, (hal. 146 148).
  35. Islami, D., Munawaroh, F., Hadi, W. P., & Retno, A. Y. (2018). Analisis Miskonsepsi Siswa pada Mata Pelajaran IPA Materi Listrik Statis Menggunakan Four Tier Test. Science Education National Conference.
  36. Islami, Diana; Munawaroh, Fatimatul; Hadi, Wiwin Puspita; Wulandari, Ana Yuniasti Retno. (2018). Analisis miskonsepsi siswa pada mata pelajaran IPA materi listrik statis menggunakan four tier test. Science Education National Conference.
  37. Isnaini, I., Utami, W. B., Susongko, P., & Lestiani, H. T. (2019). Estimation of college students' ability on real analysis course using Rasch model. Research and Evaluation in Education, 5(2), 95-102.
  38. Jubaedah, D. S., Kaniawati, I., Suyana, I., Samsudin, A., & Suhendi, A. (2017). Pengembangan Tes Diagnostik Berformat Four Tier untuk Mengidentifikasi Miskonsepsi Siswa pada Topik Usaha dan Energi. Prosiding Seminar Nasional Fisika SNF 2017.
  39. Kaltakci-Gurel, Eryilmaz, A., & McDermott, L. C. (2017, April 18). Development and application of a four-tier test to assess pre-service physics teacher's misconceptions about geometrical optics. Research in Science &Technological Education.
  40. Karolina, A. (2016). Pengembangan dan Validasi Tes Virtual Berpikir Tingkat Tinggi (Level Analisis) pada Materi Asam Basa. repository upi.
  41. Kironoto, B. A. (2018). Statika Fluida. Yogyakarta: Gadjah Mada University Press. Koçak, D. (2020). Investigation of Rater Tendencies and Reliability in Different Assessment Methods with Many Facet Rasch Model. International Electronic Journal of Elementary Education, 12(4), 349-358.
  42. Krillin, A. (2020). Dynamic modification of the G Rasch's logistic model for the tasks of examination the engineers' knowledge. IOP Conference Series: Materials Science and Engineering.
  43. Kurniadi, H. (2016). Pengembangan dan Validasi Virtual Tes Berbasis Multimedia untuk Mrngukur Kemampuan Pembuatan Keputusan pada Materi Larutan Penyangga.
  44. Liu, G., & Fang, N. (2016). Student Misconceptions about Force and Acceleration in Physics and Enineering Mechanics Education. International Journal of Engineering Education, 32(1 (A)), 19-29.
  45. Masidjo. (1995). Penilaian Pencapaian Hasil Belajar Siswa di Sekolah. Yogyakarta: Kanisius.
  46. Maulana, S., Supriyati, Y., & Astra, I. M. (2020). DEVELOPMENT OF PHYSIC PROBLEMS FOR THE FINAL ASSESSMENT IN HIGH SCHOOL LEVEL WITH RASCH MODEL ANALYSIS. Unnes Science Education Journal, 9(1), 22-29.
  47. Mayer, R. E. (2001). Multimedia learning. Newyork: Cambridge University. Mokshein, S. E., Ishak, H., & Hishamuddin, H. (2019). THE USE OF RASCH MEASUREMENT MODEL IN ENGLISH TESTING. Cakrawala Pendidikan, 38(1), 16-32.
  48. Mursidi, A., & Soeharto, S. (2016). AN INTRODUCTION: EVALUATION OF QUALITY ASSURANCE FOR HIGHER EDUCATIONAL INSTITUTIONS USING RASCH MODEL. Journal of Education, Teaching and Learning, 1(1), 1-6.
  49. Nielsen, T., & Kreiner, S. (2018). Measuring statistical anxiety and attitudes toward statistics: The development of a comprehensive Danish instrument (HFS-R). Cogent Education, 15(1), 1-19.
  50. Nooritasari, D. D., Kusairi, S., & Wisodo, H. (2020). The exploration of mechanistic reasoning ability for high school students in static fluids. Journal of Physics: Conference Series.
  51. Nurhudaya, N., Taufik, A., Yudha, E. S., & Suryana, D. (2019). The Raven's Advanced Progressive Matrices in Education Assessment with a Rasch Analysis. Universal Journal of Educational Research, 7(9), 1996-2002.
  52. Planinic, M., Boone, W. J., Susac, A., & Ivanjek, L. (2019). Rasch analysis in physics education research: Why measurement matters. PHYSICAL REVIEW PHYSICS EDUCATION RESEARCH, 15(2), 1-14.
  53. Pratiwi, N. J., Ramalis, R., & Samsudin. (2019). The three-tier Diagnostic Instrument; Using Rasch Analysis to Develop and Assess K-10 Student's Conception on Force Concept. International Research Conference on Science and Technology, 654-663.
  54. Purwanto, M. G., Suhandi, A., Costu, B., Samsudin, A., & Nurtanto, M. (2020). Static Fluid Concept Inventory (SFCI): A Gender Gap Analysis using Rasch Model to Promote a Diagnostic Test Instrumen on Student's Conception. International Journal of Advanced Science and Technology, 29(6).
  55. Quint, F., Sebastian, K., & Gorecky, D. J. (2015). A mixed-reality learning environment. Springer, 75, 43-48.
  56. Rabbitt, M. P. (2016). A Note on the Usefulness of the Behavioural Rasch Selection Model for Causal Inference in the Social Science. Journal of Physics: COnference Series.
  57. Rahmani, B. D. (2018). Differential item functional analysis on pedagogic and content knowledge (PCK) questionnaire for Indonesian teachers using RASCH model. IOP Conf. Series: Journal of Physics, 1-7.
  58. Rawh, P., Samsudin, A., & Nugraha, M. G. (2020). Pengembangan Four-Tier Diagnostik Test untuk Mengidentifikasi Profil Konsepsi Siswa pada Materi Alat-alat Optik. WaPFi, 5, 84-89.
  59. Ray, S., & Srivastava, S. (2020). Virtualization of science education: a lesson from the COVID 19 pandemic. Journal of Proteins and Proteomics, 77-80.
  60. Rochmad. (2012). Desain Model Pengembangan Perangkat Pembelajaran Matematika. Jurnal Kreano, 3(1).
  61. Rohmani, R., Sunarno, W., & Sukarmin, S. (2016). Pengembangan Media Pembeljaran Fisika Berbasis Multimedia Interaktif Terintegrasi dengan LKS Pokok Bahasan Hukum Newton tentang Gerak kelasl X SMA/MA. Inkuiri, 152-162.
  62. Rosyidah, T. H., Firman, H., & Rusyati, L. (2017). Comparing Science Virtual and Paper-Based Test to Measure Students' Critical Thinking based on VAK Learning Style Model. Journal of Physics: Conference Series.
  63. Rusyati, L. (2019). Validation of science virtual test (SVT) to assess 9th grade studens' critical thinking on living things and environmental sustainability theme. Journal of Physics: Conference Series.
  64. Samsudin, A., Suhandi, A., Rusdiana, D., Kaniawati, I., & Costu, B. (2017). Promoting conceptual undesrtanding in magnetic field concept through interactive conceptual instruction (ICI) with PDEODE*E tasks. Advance Science Letters, 23(2), 1205-1209.
  65. Saukani, M. (2015). Pengembangan dan Validasi Virtual Test untuk Mengukur Kemampuan Pembuatan Keputusan (Decision Making) siswa pada Materi Larutan Asam Basa. Bandung: UPI.
  66. Scherer, R., Koppelt, J. M., & Tiemann, R. (2014). International Journal of STEM Education.
  67. Schumaker, R. M., Smith, R. E., & Busch, M. J. (1995). Using Item Mean Squares to Evaluate Fit to the Rasch Model. Annual Meeting of the American Educational Research Association. American Educational Research Association.
  68. Sholihat, F. N., Samsudin, A., & Nugraha, M. G. (2017). Identifikasi Miskonsepsi dan Penyebab Miskonsepsi Siswa Menggunakan Four-Tier Diagnostic Test pada Sub Materi Fluida Dinamik: Azas Kontinuitas. Jurnal Penelitian & Pengembangan Pendidikan Fisika, 3(2), 175-180.
  69. Siuki, H. A., Tehrani, H., Aval, M. G., Ebrahimipour, H., Jafari, A., & Shahroodi, M. V. (2020). Psychometric Properties of a Questionnaire on Brucellosis Prevention Behaviors Based on the PRECEDE Model Among Rural Farmers and Their Family Members. Risk Management and Healthcare Policy, 13, 539-548.
  70. Streveler, R., T, L., R, M., & P, S. (2008). Learning conceptual knowledge in engineering: Overview and future research direction. Journal of Engineering Education, 97(3), 279 294.
  71. Sugiarti, T. (2014). Pengembangan Instrumen Berpikir Kritis pada Mata Pelajaran Fisika SMA. Prosiding Seminar Nasional Fisika, (hal. 83-88).
  72. Sugiyono. (2012). Sugiyono. (2013). Metode Penelitian Pendidikan. Bandung: CV Alfabeta.
  73. Sumintono, B., & Widhiarso, W. (2014). Aplikasi Model Rasch untuk Penelitian Ilmu-ilmu Sosial. Bandung:
  74. Trim Komunikata. Supahar, & Bashooir, K. (2018). Validitas dan reliabilitas instrumen asesmen kinerja literasi sains pelajaran Fisika berbasis STEM. Jurnal Penelitian Dan Evaluasi Pendidikan. 22(2), 168-181.
  75. Grasindo. Suranata, K., Rangka, I. B., Ifdil, I., Ardi, Z., Susiani K, Prasetyaningtyan, W. E., . . . Rahim, R. (2018). Diagnosis of students zone proximal development on math design instruction: A Rasch analysis. IOP Conf. Series: Journal of Physics, 1-6.
  76. Susongko, P. (2016). Validation of science achievement test with the rasch model. Jurnal Pendidikan IPA Indonesia, 5(2), 268-277.
  77. Thiagarajan, S., Semmel, D. S., & Semmel, M. I. (1974). Instructional Development for Training Teacher of Exceptional Children; A source Book. National Centre for Inprovement of Educational System (DHEW/OE).
  78. Thiagarajan, S., Semmel, D. S., & Semmel, M. I. (1974). Instructional Development for Training Teacher of Exceptional Children; A source Book. National Centre for Inprovement of Educational System (DHEW/OE).
  79. Tipler, P. A. (2001). Fisika untuk Sains dan Teknik Edisi 3 jilid 1; alih bahasa. Jakarta: Erlangga.
  80. Tornabene, R. E., Lavington, E., & Nehm, R. H. (2018). Testing validity inferences for Genetic Drift Inventory scores using Rasch modeling and item order analyses. Evolution: Education and Outreach, 11(6), 1-16.
  81. Tumanggor, A. M., Supahar, Kuswanto, H., & Ringo, E. S. (2020). Using four-tier diagnostic test instruments to detect physics teacher candidates' misconceptions: Case of mechanical wave concepts. IOP Conf. Series: Journal of Physics, 1-8.
  82. Tvenge, N., Ogorodnyk, O., Ostbo, N. P., & Martinsen, K. (2020). Added value of virtual approach to simulation-based learning in a manufacturing learning factory. CIRP Conference of Intelligent Computation in Manufacturing Engineering, CIRP ICME, 36-41.
  83. Umar, R. (2013, Juli 2). REVIEW TENTANG VIRTUALISASI. Jurnal Informatika, 7.
  84. Urbaningrum, R. (2016). Mekanika Fluida dan Hidrolika. Dalam Mekanika Fluida dan Hidrolika CIV-106.
  85. Valantika, L., Firman, H., & Nahadi. (2017). Development virtual test of reaction rate based visual perceptual skills to measure students' mastery concept. Journal of Physics: Conference Series.
  86. Veas, A., Castejón, J.-L., Gilar, R., & Miñano, P. (2017). Validation of the Spanish adaptation of the School Attitude Assessment Survey-Revised using multidimensional Rasch analysis. anales de psicología, 33(1), 74-81.
  87. Waltz, C. F., Strickland, O. L., & Lenz, E. R. (2010). Measurement in Nursing and Health Research. New York: Springer Publishing Company.
  88. Wang, H. (2019). Application of Virtual instrument in demonstration experiment of Electronic Technology. Journal of Physics: Conference Series.
  89. Wang, W. C., & Chen, C. T. (2018). Item Parameter Recovery, Standard Error Estimate, and Fit Statistics of the Winsteps Program for the Familu of Rasch Model. International Journal of Libraries and Information Studies, 65(3), 43-57.
  90. Wibowo, F., & Iswanto, B. (2019). Designing MOOCS with Virtual Microscopic Simulation (VMS) for increasing of student's levels of understanding. Jornal of Physics: Conference Series.
  91. Wilantika, N., Khoiri, N., & Hidayat, S. (2018). Pengembangan Penyusunan Instrumen Four tier Diagnostic Test untuk Mengungkap Miskonsepsi Materi Sistem Ekskresi di SMA Negeri 1 Mayong Jepara. Jurnal Phenomenon, 08.
  92. Wilson, R. F., Pan, W., & Schumsky, D. A. (2012). Measurement and Evaluation in Counseling and Development. 45(3), 197-210.
  93. Xu, R. H., Wong, E. L.-y., Lu, S. Y.-j., Zhou, L.-m., Chang, J.-h., & Wang, D. (2020). Validation of the Toronto Empathy Questionnaire (TEQ) Among Medical Students in China: Analyses Using Three Psychometric Methods. Frontiers in Psychology, 11, 1-11.
  94. Yasin, S. N., Yunus, M. F., & Ismail, I. (2018). The use of rasch measurement model for the validity and reliability. Journal of Counseling and Educational Technology, 1(2), 22-27.
  95. Yuberti, Suryani, Y., & Kurniawati, I. (2020). FOUR-TIER DIAGNOSTIC TEST WITH CERTAINTY OF RESPONSE INDEX TO IDENTIFY MISCONCEPTION IN PHYSICS. Indonesian Journal of Science and Mathematics Education, 3(2), 245-253.
  96. Yunos, J. M., Mukhtar, M. I., Alias, M., Lee, M. F., Tee, T. K., Rubani, S. N., . . . Sumarwati, S. (2017). VALIDITY OF VOCATIONAL PEDAGOGY CONSTRUCTS USING THE RASCH MEASUREMENT MODEL. Journal of Technical Education and Training, 9(2), 35-45.
  97. Zulfikar, S. A., & Saepuzaman, D. (2017). Pengembangan terbatas Tes Diagnostic Force Concept Inventory Berformat Four Tier test. Jurnal Whana Pendidikan Fisika, 2(1), 43-49.