The exam that measured the tool

A Brown economics professor gave his class a take-home midterm and got the best scores of his career. The average was 96 out of 100; forty of eighty-six students turned in perfect papers. In a hard course whose average had run between 65 and 80 for years — one that in some semesters drew only eight students — that wasn't a triumph. It was a reading, and Roberto Serrano knew how to read it.

The answers were correct but slightly off, he said — a convoluted style that came back looking familiar when he and his graduate students fed the questions to ChatGPT. So he made the final in-person and offered the class a deal: if the two exams looked alike, the midterm would stand. Twenty-seven students left rather than take him up on it — eighteen dropped the course, nine skipped the final. Of those twenty-seven, twenty-two had scored a perfect 100 on the midterm. Among the students who actually sat down, the average fell to 48.

Treat those two numbers as what they are: measurements of two different things. The 96 measured a language model. The 48 measured the students. The take-home didn't fail to catch cheating so much as it quietly ran the exam on the wrong subject — it scored ChatGPT and printed the result under a person's name. The proctored final scored the person. The distance between them, forty-eight points, is the size of the thing that had been borrowed.

This is not one eccentric class. A survey this spring found nearly 30 percent of Princeton students admitted to using AI on at least one exam or assignment. What Brown adds is the control group — the same students, the same material, measured once with the tool and once without. You rarely get the before-and-after that cleanly.

The most honest figure in the episode isn't either average. It's the twenty-two. Those students had a perfect grade in hand and one chance to defend it by doing the work in a room. They chose to forfeit it instead. That's a revealed preference, and it's unambiguous: they would rather surrender the 100 than let it be checked against what they could actually do. They already knew the number wasn't theirs.

A grade is a claim — that a particular person can do a particular thing. When the tool takes the exam, the claim comes loose from the person, and nobody reading the transcript later can tell which is which. That's the cost, and it doesn't land on the professor. It lands on every future reader of the record who has to trust it, and on the student who traded the learning for the mark and now can't recover the difference. Serrano's warning is aimed exactly there:

The line reads as thunder, but the mechanism underneath it is quiet. Nobody chooses to become an idiot; you choose a shortcut, once, under deadline, and then again, and the choice is never dramatic enough to feel like the decision it adds up to. The number to keep isn't the 96 or the 48. It's the twenty-two who, offered a single chance to prove the grade was theirs, stood up and left the room.