Mar 2022
In a recent post, Parrhesia suggested that course grades should be 100% determined by performance on a final exam—an exam that could be taken repeatedly, with the last attempt being the course grade. (See also the discussion at r/slatestarcodex.) The idea is that grades are supposed to measure what you know, and if you do well on a final, then you know the material.
Ha. Haha. Hahahahahahahaha.
Now, I sympathize with this proposal. I largely agree with the central claim that this would be more accurate than grades based on a mixture of homework and quizzes and whatever.
And yet—I suspect this proposal hasn’t seen much contact with people who’ve actually taught classes. Systems with humans in them behave in funny ways, which means there are other considerations beyond accuracy.
I don’t mean to suggest that things are optimal the way they are. But we should at least understand how they came to be, so let’s follow Chesteron’s fence for a while, shall we?
Here are some things that I hated as a student. At the time, I thought my teachers didn’t understand or care how terrible they were. I now see them as the result of structural forces.
Assignments with agonizingly precise instructions
As a student, I often got assignments that a sane person would write as:
Assignment:
- Build a temperature monitor circuit.
- Test it to prove it works.
- Write a report.
But they wouldn’t be written like that. They would be written like this:
Assignment:
- Step 1: Get a 400-point breadboard. (YOU MAY NOT USE AN ALTERNATE BREADBOARD. NO EXCEPTIONS.)
- Step 2: Write by hand the statement “I have used a 400-point breadboard. In particular, I have not used a 630-point or 830-point breadboard and understand that all credit on this assignment is forfeit if I did.” Sign and date below. (MANDATORY: ASSIGNMENT WILL RECEIVE NO CREDIT IF SKIPPED.)
- Step 3: Place the breadboard on a table with the long axis facing at an angle of 22 degrees from magnetic north.
- Step 4: Take a picture of the breadboard next to a compass and your student ID. (MANDATORY: ASSIGNMENT WILL NOT BE GRADED IF PICTURE IS NOT INCLUDED.)
…
- Step 134: Format your measurements in a 13 x 9 table. Each entry must be written with 4 points of precision. Format this table in 12-point Palatino font with a font-weight of 500. (MANDATORY: AN ALTERNATE SERIF FONT WILL HAVE A 10 POINT PENALTY AND THE ASSIGNMENT WILL BE RETURNED UNGRADED IF A SANS-SERIF FONT IS USED.)
Why? Why so much pain?
Here’s how this happens. A sweet optimistic teacher begins their career. Remembering their own agony, they give out the nice version of the assignment. And when students submit their solutions, most are fine. But some are an assault on reason, with every word of the assignment creatively misinterpreted. It was never stated which temperature circuit to build or how to prove it works or what level of explanation was necessary. And who’s to say what “build” means?
The teacher protests that students should be “reasonable”. And most of the students are amazingly gracious and drop the issue. But some don’t, and they keep complaining and asking for regrades, and if those aren’t accepted they (or their parents) contact the principal/chair/dean/ombudsperson, who are required to have an investigation.
The teacher never seriously worries they’ll get into trouble—often the investigation is a sham—and in the end they’re vindicated. But the whole thing was a huge headache and very much not what the teacher accepted an Xtra-Lite salary to spend their life doing. So, next semester they add a bunch of clarifications. But nature finds a way: That gets misinterpreted too, so more details are added, and by the time the teacher retires you have a monstrosity that’s universally despised but almost impossible to complain about.
Grade boundary agony
As a student, I had an incredible talent for getting grades like 89.952%. I’d sometimes have conversations like this:
Dynomight: Hello, teacher! [flutters eyes] I hoped that you might find it in your heart to round that up to 90% and give me an A?
Teacher: No.
Teacher: Oh, and don’t bother submitting any previous assignments to be regraded. You get a B, that’s final, good day.
(I had some teachers who tried to avoid the issue by setting the A boundary at 89.5%. I outwitted them by earning 89.483%.)
Why no flexibility? Well, suppose you are my teacher and you decide to be flexible. Word will get out. You will quickly find you didn’t just “round me up”—you made 89.952% the new boundary for students who got 89.903% to plead to be rounded to. There is no equilibrium.
Or, suppose you don’t do rounding but you allow students to submit regrade requests after grades are posted. Well, enjoy re-grading every single assignment from every student near a boundary, plus debating the exact amount of partial credit for every wrong answer. Was it really only worth 3/10 points rather than 4/10? Enough to impact my future?
The problem is that student performance is continuous. When you are forced to discretize into a small number of bins, injustice is inevitable.
Cheating and arcane strategies
Most students don’t cheat. (Really, this seems to be true!) But some do.
The actual “victims” of cheating are extremely diffuse: It’s the very slight degradation of respect that comes to anyone with a credential from wherever the cheating happens. Cheaters are a kind of special interest group. Though their behavior hurts overall welfare (let’s assume), it’s not rational for any of the people they hurt to bother opposing them.
This puts teachers in a strange position. No one will thank you for policing cheating. Not the cheaters, not the honest students who feel inconvenienced and mistrusted, and certainly not the school administration who have to process academic dishonesty paperwork.
So what do teachers do? As far as I can tell, most follow the incentives and make little effort to stop cheating.
But some teachers are principled and are determined to police cheating anyway. For these, the best strategy is often to use “arcane methods”. Some of these are truly ingenious, but I don’t want to ruin them so I’ll talk about an obvious one: Say you suspect students are copying from each other on an exam. You can silently prepare multiple versions of the exam with “micro differences” in questions. If someone submits the correct answers to a different version of the test, they’re done.
The advantage of this is that honest students don’t even know it is happening. And cheaters can’t take countermeasures, since there’s no warning. But it leads to awkwardness. Students might ask, “Can you release the solutions?” or “Can you go over question 7 in class?” You can’t do these things, and you also can’t explain why you can’t do them.
There are many techniques like this. These end up with policies that look arbitrary. But to explain why they exist would either
- help cheaters to avoid detection, or
- make it look like the instructor doesn’t trust the class.
So you’re left with policies that seem bizarre and a teacher who will dissemble when asked to explain them.
Regrades
Here’s a story from my father. He was teaching a course for working professionals that had a large project component. He—being naive and idealistic—decided that as long as the students eventually finished the project, they had learned the material, so they should get full credit. Thus, his policy was that students could submit the project, get it graded, and then repeat this process as many times as they wanted. He knew this would mean extra work for him, but thought it would be worth it for the students.
The result, of course, was catastrophe.
To call the strategy many students took “abuse” gives no measure of their ingenuity. They realized that they could skip learning the material, and instead complete the project by running an evolutionary algorithm with my father’s grading as a reward function. Roughly speaking:
- Write down some gibberish.
- Submit it.
- Make a random change, possibly informed by feedback on the last submission.
- Resubmit it. If the grade improved keep it, otherwise revert to the old version.
- Go to 3.
It got to the point where he would hand students their assignments back, and they literally would sit in front of him and ask, what grade would I get if I made this change?
And after all this, how do you think his course evaluations looked regarding grading? Positive or negative?
Homework grades and deadlines
So back to Parrhesia’s proposal. Why do we have homework grades? And why penalize students for submitting homework late, when this has nothing to do with their level of understanding?
Now, some classes can’t be graded based on a final. (Say, painting.) But for the sake of argument, say you’re teaching math and say there’s a final exam that perfectly measures student understanding.
You might think that that you can drop homework scores from the final exam. But just try it. Here’s what will happen:
- Like most other humans, your students will be lazy and fallible.
- So many of them will procrastinate and not do the homework.
- So they won’t learn anything.
- So they will get a terrible grade on the final.
- And then they will blame you for not forcing them to do the homework.
Similarly, what’s the point of deadlines? As long a student learns the material before the semester is up, that proves they’ve learned it, right?
Well, you can probably guess what happens when you get rid of deadlines: Many students will do almost nothing until the end of the semester, then get overwhelmed and flame out, and then blame you for not imposing deadlines on them.
You could even try a grading policy like this:
GRADE = MAX(FINAL, ½(HW + FINAL)).
Like getting rid of deadlines, that grading scheme is in theory strictly better for students than just using ½(HW + FINAL). But I predict that change will in reality make grades go down because second-order effects exist.
And you know what? When the students blame you, maybe they are right. The teacher is supposed to use their experience to help students learn. Shouldn’t you help the actual imperfect humans in front of them, rather than imagining a bunch of perfectly rational Platonic objects?
Participation grades
How infantilizing, right? Surely what matters is if a student understands things, not if they ask questions in class?
Well, one reason for participation credit is to give an extra incentive for behavior that helps students anyway, like with deadlines or homework grades above. But there’s a more interesting reason, too.
Participation credit solves a collective action problem. A class is better for everyone if lots of students ask lots of questions. But asking questions is frightening. The best thing for every individual student would be that they sit back while everyone else asks lots of questions. Participation credit helps to internalize positive externalities.
Again, I’m not claiming things are optimal as they are. More enlightened policies are surely possible. But these must be carefully designed to avoid the large known downsides.
For example: Since grade boundaries cause injustice, why don’t we get rid of them? Or if that’s too much, why don’t we at least make transcripts also show unrounded grades?
Or: The controversy about homework grades and deadlines stems from the dual use of grades as measurements and incentives. The incentives are imposed on everyone, even the people who don’t need them.
We face many situations with this kind of tension.
- Recreational fentanyl is illegal, even for people who could use it responsibly.
- Doritos® loaded Cool Ranch® cheese snacks are legal, even for people who can’t control themselves and wish they were banned.
But sometimes this tension has a solution: Opt-in Odysseusing. For example, in some places, gambling addicts can put their name on a list and ask casinos to ban them from entering.
So, some students are responsible enough to manage their time without the need for homework grades or deadline penalties. How can we help them without hurting the majority who need these things?
We could change from the current “mandatory Odysseus” regime to an “optional Odysseus” regime: On the first day of class, offer students an irrevocable choice: They can have homework and deadlines imposed on them, or not. Perhaps the students who need deadlines would learn to opt for them and others could live freely.
Of course, this wouldn’t work. What would actually happen is that some students wouldn’t be interested in lessons from Homer and would want to revoke their irrevocable choice and would complain bitterly that it was offered in the first place and the teacher would need to explain their cruel and unusual policy to annoyed principals and parents. But at least you’d be failing in a new and groundbreaking way.
(I encouraged Parrhesia to respond to all this, I’ll add a link when it’s available.)