I recently read an interesting paper by Ido Millet, a professor of Management Information Systems at Penn State entitled, “Improving Grading Consistency through Grade Lift Reporting.” The premise is simple: when you have many instructors teaching sections of a class, some will grade easy, some hard and the rest somewhere in between. How do you measure this? Millet proposes using “grade lift,” which is calculated as:
(grade lift) = (average final grade in the class) – (average GPA of the students in the class).
For example, if the average final grade in the class is 3.5 and the average GPA of the students in the class is a 3.1, the instructor has a positive grade lift of 0.4. Grade lift can be negative if most of the students finish the class with a grade lower than their GPA would predict. Instructors with a positive grade lift presumably grade harder than their colleagues with a negative grade lift. Admittedly, grade lift is an imperfect measure. An exceptionally good instructor might regularly inspire their students to perform above expectation. But exceptional instructors are, well, exceptional.
Let’s take a look at the grade lift for a Temple General Education class with multiple sections. The amount of grade lift is presented by instructor, sorted from low to high.
Instructor lift ranges over more than an entire grade, which suggests (but does not prove) that students taking the same class with the same syllabus and assignments may receive different grades for the same performance depending on who they get as an instructor. I’m sure most students would tell you this is not news.
Millet’s solution to encourage greater consistency is simple: share grade lift data with the instructors. No instructor is told they have to change, but those who discover their grading practices are far from the norm are likely to reflect. Millet found that the extreme graders on both ends tended to regress back toward the mean in subsequent semesters.
Ido Millet. (2010). Improving Grading Consistency through Grade Lift Reporting. Practical Assessment, 15(4), 1-8.