Interview: “Previous examination formats will be rendered useless by using AI”
Are students cheating in performance assessments with AI? If so, you should save yourself a rabbit and hedgehog race and rethink, explains Bernhard Gmeiner.
(Image: The American Explorer/Shutterstock.com)
Since AI tools such as ChatGPT have become readily available to children and young people, some of them also use them to complete their schoolwork. As a result, it is becoming increasingly difficult to determine what is actually a pupil's performance.
Austrian teacher Bernhard Gmeiner believes that the examination formats used to date no longer do justice to this development. He has been studying the effects of digital offerings and AI tools on the school system for years, and has now produced a free handbook on AI-resistant examination formats. In an interview, heise online asked him why he did this, what problems he believes established examination formats now have and what he would like to see in the future.
At the beginning of June, you wrote a blog post for standard.at about exam culture in the age of AI. In it, you called for more thought to be given to this topic and for exams to be adapted. Now you have written a freely available handbook for AI-resistant exam formats. Did you set yourself this task a few weeks ago?
In discussions with colleagues from Germany and Austria, I realized that this is the next big challenge in the education sector because we are noticing in everyday life how previous examination formats are becoming useless due to the use of AI. There is also a great need for change here, as we are observing the associated skills simulations and skill-skipping by pupils.
The starting point for my handbook was a guide by German colleagues Falck and Flick. I find their ideas very exciting. I tried to present and discuss them for the standard. Even then, I realized that it was not yet a finished solution. However, I do see the need for guidelines for teachers – and I realize this again and again in the workshops I give for teachers. That's why I want to start or promote a discussion regarding AI-resistant exams. The publication of my handbook is part of this.
Is the topic not being sufficiently addressed by official bodies – i.e., school authorities and ministries –? Are there recommendations for action or new guidelines?
There are guidelines, but they remain vague and do not change anything fundamental. As far as I know, the situation is similar in Germany. Two worlds are colliding, so to speak. On the one hand, we have a technological revolution through AI – I think the word revolution is very appropriate here. New tools are coming out all the time; the speed is insane! And this is changing society – quickly! And on the other hand, there is a ponderous, heavy steamer called education policy. It is very difficult to react quickly to rapid changes in this structure.
Instead of rethinking teaching and examinations in terms of AI tools, from my perspective the focus is currently more on how cheating with AI can be better prevented. How can this be exposed? So it's more of a defensive stance than a proactive approach that looks at the added value and develops solutions to accompany the process.
In one of your blog posts, you commented on attempts to expose homework done with AI. For you, this is a wasted effort.
Although I understand the initial reaction of colleagues to want to find out exactly whether a child has used an AI to do their homework, I think this is the wrong approach. Some use cloaking software such as ZeroGPT for this purpose, but such tools provide uncertain results and regularly generate false positives, i.e. cases in which human-generated texts are falsely marked as AI-generated.
At the same time, some students deliberately include small errors or stylistic breaks in their AI texts to further disguise their origin. I don't think it makes sense to fight this battle. I also think this interplay between teachers and students is a symptom of an examination culture that is no longer in keeping with the times.
What do you mean by that?
For me, the defensive struggle against AI tools in schools is a symptom of how we have designed schools in recent decades – how we test performance, what formats we use for this and what our assessment schemes look like. What do we even consider to be an achievement?
For me, for example, interaction is a key focus – of the student-teacher relationship. And that's a huge challenge because I teach around 150 students directly. For me, this relationship is based on honesty and is ideally process-oriented. However, this is not reflected in the way we are supposed to test in schools. The assessments are to be completed to a certain point, in a certain format – mostly class tests. Anything that goes beyond this is more difficult to assess in everyday school life and to integrate into existing processes – but this is exactly where the focus should be.
In your handbook, you propose precisely this approach to process-oriented work and assessment. You do not want to exclude AI, but rather promote a culture of inclusion and transparency. Some formats continue to manage without AI at all, while others explicitly allow AI. How AI is used, reflected on and documented is then assessed or evaluated. However, this is a process that is not completed with a class test in the course of two lessons, for example.
Yes, it is moving away from that. Of the four pillars that I described in my handbook – orality, process, context, and collaboration – process is the biggest change. Orality is already a part of exam culture, but it is becoming more important with AI tools. In my model, it becomes more similar to a defense of a written paper as in university and does not require the inclusion of AI. “Context” means that the use of AI can also be limited by giving students tasks that are more in line with their real lives, biographies and specific locations – These are spaces of experience or points of reference that AI cannot simply generate. Collaborative work is part of process-based work, but in community. This requires several modern skills, including the use of AI. How do I collaborate with it?
For teachers, the switch to checking work processes would be a closer look at individual learning paths. Teachers become more like facilitators. And students show how well they handle new tools, how well they can classify and process information.
Videos by heise
This form of process-oriented, transparent work is actually what has always been expected of academic work. Topics are narrowed down, theses established, assumptions made clear, lists of sources and tools compiled.
Yes, and so that I as a teacher can understand how the students are working, I can even use tools myself that directly create documentation about prompts and sources. Process-oriented and AI-accompanying writing with functions such as those offered by Fellofish and Fobizz, for example, go in this direction.
If the work processes are made transparent and also reflected on along the way, this is part of the performance – or, for me as a teacher, an opportunity to understand the process and discuss improvements.
The guidelines of the Austrian Ministry of Education describe this so roughly that it is not enough. It is important to know not only a prompt that has been written, but how it is constantly being adapted. A multi-stage conversation is held with the AI, and this is relevant for a performance assessment, but also for the monitoring of skills development. In order for children and young people to be able to use AI tools confidently, they need to engage with them in detail.
