Reliability and Validity of an Automated Model for Assessing the Learning of Machine Learning in Middle and High School: Experiences from the “ML for All!” Course
Reliability and Validity of an Automated Model for Assessing the Learning of Machine Learning in Middle and High School: Experiences from the “ML for All!” Course
Author(s): Marcelo Fernando Rauber, Christiane Gresse von Wangenheim, Pedro Alberto Barbetta, Adriano Ferreti Borgatto, Ramon Mayor Martins, Jean Carlo Rossa HauckSubject(s): Education, School education, ICT Information and Communications Technologies, Pedagogy
Published by: Vilniaus Universiteto Leidykla
Keywords: K-12; middle and high school; Machine Learning; Artificial Intelligence; neural network; image classification; assessment; evaluation;
Summary/Abstract: The insertion of Machine Learning (ML) in everyday life demonstrates the importance of popularizing an understanding of ML already in school. Accompanying this trend arises the need to assess the students’ learning. Yet, so far, few assessments have been proposed, most lacking an evaluation. Therefore, we evaluate the reliability and validity of an automated assessment of the students’ learning of an image classification model created as a learning outcome of the “ML for All!” course. Results based on data collected from 240 students indicate that the assessment can be considered reliable (coefficient Omega = 0.834/Cronbach’s alpha α = 0.83). We also identified moderate to strong convergent and discriminant validity based on the polychoric correlation matrix. Factor analyses indicate two underlying factors “Data Management and Model Training” and “Performance Interpretation”, completing each other. These results can guide the improvement of assessments, as well as the decision on the application of this model in order to support ML education as part of a comprehensive assessment.
Journal: Informatics in Education - An International Journal
- Issue Year: 23/2024
- Issue No: 2
- Page Range: 409-437
- Page Count: 29
- Language: English