Hell, back in my day, teachers were even very picky about what kind of calculator you could use. And if it was a graphing calculator, you had to show them yourself wiping the memory at the beginning of the test.
(Except for one algerbra teacher, who was really cool about it. He’d allow custom programs to stay on the calculator if you programmed it yourself. On the theory that if you can write a computer program that reliably solves these math problems, then you must have a very good understanding of how to solve these math problems. And, yes, I was one of the few kids who actually did that. Ah, writing my own custom software for the TI-83 on the TI-83, because that seemed easier than actually doing the math problems by hand … good times.)
but there’s a tendency of focusing more on the work during the semester than in the exam itself
Perhaps this tendency needs to be reversed?
If you have one big exam (or a few of them spread over the year) that it’s impossible to use LLM help for and those exams carry enough weight to make the student fail the class if they completely bomb it … then you’ll be stopping the LLM-cheaters dead in their tracks. Sure, they can be lazy and do much of their coursework that way, but if they’re being lazy like that, they’re likely not actually learning anything, and that will show up during the big exams. And when they fail those big exams and then fail the class, hopefully they’ll learn their lesson about relying on LLMs to get them through classes.
Right, and that’s what’s going to have to change: a bigger focus on things like in-person tests (including in-person bluebook essays), oral presentations/thesis defenses instead of other project deliverables, etc.
A teacher I know says it is easy to determine if a student wrote their paper if you interview them about it. You’re right that automated methods are risky.
That’s it. As a teacher who has been dealing with this in the last 2-3 years, the only reliable way I have found is to do short interviews.
Students hand in their work, I grade it, then I ask them verbally a few easy questions about what they mean in specific sections of their work. How they score on these questions is used as a coefficient that I apply on the grade to get the final score.
So they can use LLMs, but they have to understand its output.
As someone who works in ed tech these days, I’m kind of down for them as a study tool. For example, synthesizing notes and turning them into flashcards, practice tests, etc. I find that stuff to be suuuper handy if I’m trying to learn something.
But for cheating, yah, fuck that noise. A lot of classes are moving back to pencil and paper because of this, and I totally support that.
Exactly. Taking notes in class during a lecture. Copying something the instructor wrote on the board. This is all part of the learning process. The act of doing these things helps you learn.
The only skills or learnings I really seem to have retained from University are the ability to collect, and collate information and then apply it to a problem. The actual information collected and problems solved are lost to me now.
I’m also in this camp. That said, I’ve learned that everyone studies differently, and some people learn by synthesizing, and others do better when they have something structured from the jump.
I often run to weird little study teams where one person studies by taking everyone’s notes and creating sets in something like quizlet, and someone else studies by using the flashcards / tests that get spit out.
I dunno, they’re here to stay. Cat’s out of the box. Educators and education need to adapt. In person assessment is probably the ideal way to gauge progress and learning, but due to resources I don’t see it being practical.
People keep saying this as though it’s true. The odds that this current era of free and ubiquitous access to these frontier LLMs lasts forever are pretty slim.
Already, very few middle schoolers have the tech savvy to self-host anything. If it’s not a tablet, they have trouble using it.
Add to that the possibility that the data center run on memory and processors could mean that local computing power will disappear, to be replaced with devices like Chromebooks that use corporate cloud services for everything.
You can’t run anything like a frontier model on a self hosted solution. To get anywhere close you’d have to spend thousands of dollars on hardware which obviously isn’t free, or even a viable solution for the vast majority of people, let alone these students. And the quality of output you’d get from a model running on off the shelf consumer hardware like a MacBook is much more noticeably AI generated and trivial for AI detection tools to flag.
Except the whole point of education is to LEARN how to do it without these tools. If you’re just turning your brain off and handing in the output, you are literally missing the point.
It’s like using calculators on steroids. There are times to use calculators and times to force mental math. You can teach kids AI literacy and usage habits, but letting them just use no thinking makes the entire exercise pointless. We might as well close schools, because having the AI generated your math homework or essay is fucking pointless.
You wildly misunderstood my post. Firstly, I’m not suggesting students turn their brains off. Secondly, how you learn isn’t relevant to the demonstration and application of that knowledge, which is precisely why I said in person assessment is the optimal way. You ask the student and watch them, live. This is how you defend a thesis, in front of a live panel. No tools, no cheating, just your knowledge.
What I am suggesting is that the system should adapt to the reality of the technology.
Education needs to change. Including punishment for using LLMs.
Before you can punish for using LLMs, you need to be able to reliably detect the use of LLMs, including guarding against false positives.
Current AI checkers are woefully inadequate and prone to errors.
You can tell they’re using an LLM if they have a computer out during the pen-and-paper test.
How is that allowed?
Hell, back in my day, teachers were even very picky about what kind of calculator you could use. And if it was a graphing calculator, you had to show them yourself wiping the memory at the beginning of the test.
(Except for one algerbra teacher, who was really cool about it. He’d allow custom programs to stay on the calculator if you programmed it yourself. On the theory that if you can write a computer program that reliably solves these math problems, then you must have a very good understanding of how to solve these math problems. And, yes, I was one of the few kids who actually did that. Ah, writing my own custom software for the TI-83 on the TI-83, because that seemed easier than actually doing the math problems by hand … good times.)
Not US, but there’s a tendency of focusing more on the work during the semester than in the exam itself
LLMs are going to be a massive headache for me when they get older
Perhaps this tendency needs to be reversed?
If you have one big exam (or a few of them spread over the year) that it’s impossible to use LLM help for and those exams carry enough weight to make the student fail the class if they completely bomb it … then you’ll be stopping the LLM-cheaters dead in their tracks. Sure, they can be lazy and do much of their coursework that way, but if they’re being lazy like that, they’re likely not actually learning anything, and that will show up during the big exams. And when they fail those big exams and then fail the class, hopefully they’ll learn their lesson about relying on LLMs to get them through classes.
Right, and that’s what’s going to have to change: a bigger focus on things like in-person tests (including in-person bluebook essays), oral presentations/thesis defenses instead of other project deliverables, etc.
A teacher I know says it is easy to determine if a student wrote their paper if you interview them about it. You’re right that automated methods are risky.
That’s it. As a teacher who has been dealing with this in the last 2-3 years, the only reliable way I have found is to do short interviews.
Students hand in their work, I grade it, then I ask them verbally a few easy questions about what they mean in specific sections of their work. How they score on these questions is used as a coefficient that I apply on the grade to get the final score.
So they can use LLMs, but they have to understand its output.
Interviewing every student would take a lot of time.
As someone who works in ed tech these days, I’m kind of down for them as a study tool. For example, synthesizing notes and turning them into flashcards, practice tests, etc. I find that stuff to be suuuper handy if I’m trying to learn something.
But for cheating, yah, fuck that noise. A lot of classes are moving back to pencil and paper because of this, and I totally support that.
I feel like synthesising notes and turning them into flash cards how i learn things.
Exactly. Taking notes in class during a lecture. Copying something the instructor wrote on the board. This is all part of the learning process. The act of doing these things helps you learn.
The only skills or learnings I really seem to have retained from University are the ability to collect, and collate information and then apply it to a problem. The actual information collected and problems solved are lost to me now.
I’m also in this camp. That said, I’ve learned that everyone studies differently, and some people learn by synthesizing, and others do better when they have something structured from the jump.
I often run to weird little study teams where one person studies by taking everyone’s notes and creating sets in something like quizlet, and someone else studies by using the flashcards / tests that get spit out.
I dunno, they’re here to stay. Cat’s out of the box. Educators and education need to adapt. In person assessment is probably the ideal way to gauge progress and learning, but due to resources I don’t see it being practical.
People keep saying this as though it’s true. The odds that this current era of free and ubiquitous access to these frontier LLMs lasts forever are pretty slim.
How do you figure? There are open source self host able solutions right now.
Already, very few middle schoolers have the tech savvy to self-host anything. If it’s not a tablet, they have trouble using it.
Add to that the possibility that the data center run on memory and processors could mean that local computing power will disappear, to be replaced with devices like Chromebooks that use corporate cloud services for everything.
You can’t run anything like a frontier model on a self hosted solution. To get anywhere close you’d have to spend thousands of dollars on hardware which obviously isn’t free, or even a viable solution for the vast majority of people, let alone these students. And the quality of output you’d get from a model running on off the shelf consumer hardware like a MacBook is much more noticeably AI generated and trivial for AI detection tools to flag.
The monkey’s out of the bottle.
Deserved. 🤣
My point exactly. Same thing with just looking up the answer on Google or whatever
Except the whole point of education is to LEARN how to do it without these tools. If you’re just turning your brain off and handing in the output, you are literally missing the point.
It’s like using calculators on steroids. There are times to use calculators and times to force mental math. You can teach kids AI literacy and usage habits, but letting them just use no thinking makes the entire exercise pointless. We might as well close schools, because having the AI generated your math homework or essay is fucking pointless.
You wildly misunderstood my post. Firstly, I’m not suggesting students turn their brains off. Secondly, how you learn isn’t relevant to the demonstration and application of that knowledge, which is precisely why I said in person assessment is the optimal way. You ask the student and watch them, live. This is how you defend a thesis, in front of a live panel. No tools, no cheating, just your knowledge.
What I am suggesting is that the system should adapt to the reality of the technology.