Izazovi korištenja jezičnih modela OpenAI platforme u kontekstu vrednovanja studentskih rješenja projektnih zadataka iz domene računalnog programiranja
Challenges in using the language models of the OpenAI platform in the context of evaluating student solutions to project tasks from the field of computer programming

Summary/Abstract: The authors of this study have noted for years that students' motivation for continuous work is relatively low. To increase student motivation to practise, University Coledge Algebra faculty have decided to change their approach in recent years. Instead of forcing students to study for an exam on specific dates, they were given projects (assignments) to complete. The research results showed that the results obtained later improved compared to traditional exams. However, with the advent of powerful and free or very low-cost language models such as those of the OpenAI platform, teachers are faced with the challenge that students may use such models to solve a project task, which is against the code of ethics. The aim of this research is to determine what kind of results the OpenAI Platform language model can achieve when it is assigned a project task from a specific course. 2 research questions were defined. What result does the language model on the OpenAI platform achieve when it is assigned a project task from the Java Web Programming course and is there a difference in the results compared to the results achieved by the students (Q1)? Is there a difference in the results of the language model compared to the natural language in which it receives the project task (Croatian or English) (Q2)? The analysis of the results has shown that in certain cases the language model can solve the test without additional user intervention in order to obtain a good grade, although the results obtained are significantly weaker than those of the students. The language model achieves better results when the exam is given in English than in Croatian.

• Details
• Contents