Graduate Teaching Workshop
- Grading Guidelines
written by Idris Hsi, last updated August 30, 2000
Document in Microsoft Word Format
- Preserves format and appearance of grading guidelines at end.
Document in PDF Format
Associated Homework
Introduction
Grading is usually the primary responsibility of
a Teaching Assistant. As such, it will take up the most time, not just
in the process of grading but also in defending your grading methodologies
to the professor and to the student. Grades generally have been used for
the following purposes with varying degrees of validity (Wankat and Oreovicz,
Teaching
Engineering, 1993, p. 229):
-
Reward or penalty for student accomplishments
-
Communication to others about what the student has accomplished
-
Predictor of future performance
Regardless of where you stand on the value of grades, you
are being paid to give a quantitative assessment of your confidence in
this student’s understanding of the course material. Others will use this
information to determine whether a student is ready for future work, rewards,
or should be held back. So your actions can have potentially far-ranging
consequences.
The Course Instructor is ultimately responsible for the
following
-
Assessing the student’s skill level and knowledge
based on their performance in the class
-
Attaching a grade to the student’s performance throughout
the quarter.
As a grader, you are primarily responsible for
-
Scoring the student’s deliverables (exams, assignments,
presentations)
-
Helping them learn from their mistakes by your written and
oral feedback.
There are many different philosophies about how grading should
be approached but at minimum, you will need to obtain or develop a quantitative
and objective assessment criteria. This criteria measures student performance
against a set of expectations outlined by the course instructor and the
syllabus. This means careful evaluation of their work, positive and corrective
feedback, and maybe follow-up sessions to help them understand what went
wrong.
The Teaching Assistant also has a secondary responsibility
to the student. Grades are a poor feedback mechanism because they only
address the end result. Consider a class with only a midterm and a final
as the assessment criteria. A student has only one chance at a midcourse
correction to demonstrate their performance. So while the TA is not the
primary
point of instruction, through the process of grading the deliverables,
they become an essential secondary point of instruction by providing
feedback on those assignments
Teaching Assistant Responsibilities
for Grading
-
You should grade all material in a fair and consistent manner.
-
You should ensure that students know when they can expect
their assignments graded and returned, consistent with the course syllabus.
The TA should adhere to these guidelines (you’d expect no less from your
instructors).
-
You should ensure that the students receive the assignment
description and all supporting documents on time.
-
You should grade all assignments / exams with a set of grading
criteria, either developed by the course instructor or yourself, that
can be made available to students after grading to resolve any disputes.
Grading criteria should be made available to the students with the assignment
description / exam.
-
You are strongly encouraged to give students feedback about
what they did correctly and incorrectly.
General Guidelines for Equitable
Grading
Grading can become one of the bitterest topics of contention
in a class if not approached properly. Poor grading can interfere with
a student’s willingness and motivation to learn because the student’s will
not perceive the correlation between what they’ve learned and how they’re
being measured. Instead, they will feel victims of an arbitrary assessment
system that doesn’t measure what they believe they’ve achieved.
Student complaints are often thought of as noise and whining,
and some of it may fall into that category. In reality, students complain
when a strong mismatch results between the expectations of a student, honestly
based on what they believed to be their success in achieving the goals
of the class, and the assessment of their success by the TA or the instructor.
Dealing With Students Complaints
About Grading
-
"You made a mistake. The answer is correct and here’s why…"
-
"You never said we had to know this for the test / assignment
/ paper."
-
"This is what you told us to do. Why did you take off points?"
-
"My friend got the exact same answer as I did but I lost
more points for it."
-
"I did more analysis and work here than the other projects
but you took off more points because some of my analysis was wrong?"
-
"I don’t understand why you took off points here. I got the
right answer."
You should make the assumption that your students are attempting
to pass the class in good faith. This means that they are doing the reading,
the assignments, and attending lecture. When they come up with these complaints,
it’s likely that they have a good reason for them. It is your responsibility
to evaluate them on a case by case basis. After some filtering, you will
inevitably discover that some of these complaints are just bargaining.
It is much better to be proactive and keep these situations from occurring
in the first place.
The majority of the student complaints about grading can
be categorized as issues of fairness and consistency.
-
Fairness can be tied back to the class material and
expectations. Did the assignment or exam properly evaluate the student
based on the assessment goals and expectations outlined at the beginning
of the term? Given the course prerequisites and class material, could a
student accomplish the goals of the assignment? These really have to be
addressed when the assignment is created. Occasionally, you will discover
that the assignment may have exceeded the parameters of the class — beyond
just challenging the students to learn. In these cases, you may want to
adjust grades accordingly and reevaluate the design of the assignment.
-
Consistency has to do with how the grades were assigned
relative to the rest of the class. The more graders, sections, and people,
the greater the variability of the grades. This is a classic problem in
educational assessment. There are numerous confounds to being objective.
For example, fatigue can play a big factor in how you grade papers. If
you’re tired and irritable, you may tend to grade later papers harshly
compared to earlier ones. If you’re just tired or bored, you may grade
later papers with less rigor.
Some Suggestions for Grading
-
General suggestions include:
-
Give yourself enough time to grade a set of assignments.
Try to grade them in the same relative time blocks but remember to take
breaks if you get tired or start to drift.
-
If there are multiple sections across students, grade one
section/problem at a time.
-
Make sure you look over your grades, if you have time, and
check for inconsistencies.
-
Give feedback on every assignment that you grade.
-
Double check your math.
-
Always keep the larger learning context in mind - does the
student demonstrate an understanding of the material - implementation aside.
-
Don’t change a grade without a good reason! Remember
that your job is to be fair and equitable. Changing a grade means that
you may be making an exception that will treat other students unfairly
(and word will get around). On the other hand, because you are the
grader, you may feel that a special circumstance or a student’s effort
and interest in the course may warrant it. If you’re not sure, feel free
to refer the student to the course instructor.
-
You are required to have grading criteria when grading. Make
sure that these are made available to the students. For large assignments,
make sure that they see the criteria before they begin. This way, the students
can work to gain the knowledge, skills, or work that will meet your expectations.
If you’re clever about how you design the criteria, you can sneak in some
teaching goals. Note: If you alter the criteria during the course of grading,
make sure that the students know this and are aware of your rationale..
-
Remember that a grade is an abstract measurement that represents
what the students know or can do. Normal distributions exist because given
an average group of students you will have a mixture of high and low learners.
But you should watch for other distributions and statistical indicators.
Where is the mean and where did you expect it to fall? Is the distribution
bi- or tri-modal? Is the scattering completely random. This could be indicators
of poor assignment or test design or uneven grading procedures.
-
When determining how much a problem is worth or how much
the student has earned. Try to enumerate points in your grading criteria
as positive values. It’s much more objective to start with some total value
and to assign an answer a % of the total than to develop a list of negatives
and subtract from the total.
Bad Example: InsertionSort (10 points)
-5 doesn’t sort the list
-3 wrong data structure used
-6 wrong algorithm used
-1 poor style.
Clearly, you can take off more than 10 points from this
criteria if you’re
not careful.
Better Example: InsertionSort (10 points)
3 points, Algorithm does a partial sort
7 points, Algorithm works but student used a poor data
structure
4 points, Algorithm sorts but it’s not an InsertionSort.
9 points, Algorithm works but poor style
0 points, Algorithm crashes
Giving Good Feedback
-
Make sure that anything you grade has feedback on it that
will help the student understand the following.
-
what went wrong to prevent future errors.
-
what they did correctly to preserve that behavior.
-
what they can do to improve their current level of knowledge
or ability.
-
Part of your role as an educator is to encourage as well
as evaluate. When discussing assignments or exams, make sure you say something
positive about what the students have done (except in that rare case where
their work has no merit) and how they can improve. Positive feedback helps
encourage students to continue working. For example:
-
"You didn’t get the right answer because you missed a step
here. Otherwise, most of your solution was fairly well done. Next time,
just remember to follow through with this step and you’ll be fine."
-
"I like the way you set up your discussion in this paragraph.
However, I don’t follow where your example is going. I think if you came
up with a stronger case, you will have a better section here."
-
Don’t overly criticize your students. It may convince your
student that there is no use in continuing. Remember that it took you some
time to acquire the knowledge that you have now. Remember the saying that
it’s much easier to lead a donkey with a carrot than a stick.
-
Good students need encouragement also. An ‘A’ is just one
form of reward. You can help them improve their own knowledge by pointing
out where they can go with what they know or showing them additional resources
that they can access.
-
Don’t overload students with feedback.
-
Lastly, make sure your students know that you have confidence
in their ability to do the work that you’ve outlined for them.
Miscellaneous Grading Issues
I’m spending 50 hours on grading.
Is it always like this?
No and It Depends. Instructors are sometimes notorious for
giving long tests but you should also be conscious of spending too much
time on trying to catch every little error. Remember that your job is to
assess something based on a set of grading criteria. The criteria should
be the final arbiter. For example, if a program fails to pass a test, feel
free to debug it for fun but don’t spend a lot of time on it. The students
obviously turned in something that didn’t meet the criteria. Highlight
the critical sections, offer some debugging suggestions, and move on. If
you’re grading a long test or an essay, you may just need to spend that
much time. Remember that the burden of learning the material should be
on them. It’s difficult to repair a lack of understanding or effort through
feedback. If you think that you’re doing too much work, get some feedback
from other teaching assistants.
Should I accept late assignments?
If your course instructor hasn’t set up a policy for this,
you should discuss this with them. Nothing in life is absolute but you
do have to set up some standards of fairness. Does a student who turns
an assignment in late have an unfair advantage over those who turned it
in on time? Maybe. But you’re ultimately not doing them any favors by allowing
them this grace period because they’re getting behind on other assignments,
including the class reading. On the other hand, sometimes a student may
be unavoidably detained. A fair general policy is to set up a schedule
for point deductions for late assignments then deal with individual cases
as they occur. If a student has not notified you ahead of time or has been
habitually tardy, it is perfectly acceptable to refuse the assignment because
your time is valuable also.
That said, I’ve sometimes ignored the late penalties because
I’m more concerned with the quality and depth of work than I am with the
exact day that it was turned in.
Should I accept assignments that
don’t meet the submission criteria?
If your submission criteria is absolute, for readability,
compiling, or evaluation purposes, and you’ve made the submission criteria
an important part of the assignment then it’s acceptable to reject the
ones that don’t. If the submission criteria are really guidelines then
determine whether the assignment still meets the learning goals and evaluate
it based on this. It is entirely within your right to refuse or return
an assignment ungraded because it does not meet the criteria and to give
the student a chance to resubmit.
What if I don’t understand a student’s
solution but they seemed to get the correct answer?
Flag it and come back to it when you have more time. They
may be using an alternate approach or they made two wrongs and achieved
a right (rare). If you’re still not sure, check with the instructor. It
may even be useful to not assign a grade to it and have the student meet
with you. If they can convince you that they understand their solution
and they understand the "correct" approach, let them have the points.
What if I suspect academic
dishonesty?
It’s different for each type. If you suspect copying on an
exam, try and find the matching answer. Make sure you keep track of the
students sitting next to the offender. If you suspect the student copied
from other code, there are programs available in the College that will
help you to detect this. If you suspect plagiarism (very prevalent with
the World Wide Web search engines), try to follow up through the references
or through a quick Web search. In all cases, let your instructor know that
this may be a problem and show them your evidence. Do not confront the
student yourself!!!!! I can’t stress that enough. You should insulate
yourself from this unfortunate aspect of teaching. It is your job to refer
the student to your instructor. If there is sufficient evidence, the issue
gets referred to the Dean of Students who handles disciplinary action.
Program Grading
Things the Students Should Have
-
The assignment description and submission deadlines
-
Coding style requirements
-
General grading criteria (what will the program get if it
crashes, produces wrong output, etc.)
-
Assignment submission procedures
-
Access to the compiler or evaluation environment
-
Any support code necessary to complete the assignment.
-
Sample input and output files (if appropriate - you may require
them to generate their own)
Things You Should Have Prior to Grading
-
The assignment description
-
Access to the compiler or evaluation environment
-
The location of the student’s submissions
-
Test sets that match the assignment criteria.
-
Sample Data and Answer Files (or Program Running Characteristics)
-
Style guidelines (That have also been made available to students)
-
The Cheatfinder program (to catch duplicate code)
Things You May Want Prior to Grading
-
Is an autograder available? Is there a batch file that will
allow you to take the assignments and automatically generate output files?
-
A spreadsheet or grading sheet for documenting what the students
get.
Suggested Criteria Items
You’ll need to assign some point values to the following
items.
-
Does the program compile?
-
Does the program crash while running?
-
Does the program produce incorrect output?
-
Did the program meet all the assignment criteria?
-
Is there sufficient documentation?
-
Does the program have good style or design?
-
Did the student use good software engineering processes in
building or designing the code?
-
How efficient is the code?
Suggested Procedures
-
Make sure you understand the program assignment.
-
Download the code from the students and print it out. (Note:
Using ‘enscript -2rG <assignment name>’ on your Unix account will save
paper). You may want to review all the code before you start compiling
it. Some code will be definitely wrong and you can save yourself some crashing
and core dumps.
-
Develop a batch file that compiles the code and runs the
sample inputs on them if there isn’t one already. This will save you lots
of time.
-
Check the program output. It’s up to you to decide whether
the formatting was done correctly but don’t quibble too much over white
space and carriage returns.
-
Look at the code.
-
If there were errors, what caused them and did the student
document these bugs in the program header? (You should strongly encourage
students to document their own errors and take off fewer points for them
as opposed to errors you have to discover on your own).
-
Did the program meet the class style requirements - good
modularity, self-documenting code, good commenting, etc.?
-
Did the student fake the output? (Don’t laugh, you’d be surprised
what people try to pull off. We’ve seen students hard code entire graphs
into their programs).
-
Give some feedback or corrective code to the procedures that
produced wrong results. Be sure to note and encourage particularly good
code.
-
Make a list of common errors and recommendations to hand
out to students with the assignments.
-
Total the points and record them.
Sample Criteria
This is from a freshman level assignment. Programming assignments
for upper level students and graduate students will have more of the points
distributed to implementation rather than style.
Implementation of P3 : [ ] /
60
--------------------
-
followed correct algorithm [ ] /
15
-
item.java performs as expected [
] / 10
-
shelf.java performs as expected
[ ] / 10
-
warehouse.java performs as expected
[ ] / 15
-
all classes support driver [ ] /
10
Extra Credit :
------------
1. used DEBUG constant [ ] /
+3
2. used Main in each class for
incremental
programming. [ ] / up to +7
Programming Style : [ ] / 40
-----------------
1. Documentation: purpose, @param,
@return... [ ] / 10
2. Documentation: in-line [ ]
/ 5
2. Flow Mechanisms [ ] / 5
3. Subroutine Scope and Parameters
[ ] / 5
4. Data Structures [ ] / 5
5. Modularity [ ] / 5
6. Overall Style (layout, constants,
etc.) [ ] / 5
Exam Grading
Things the Students Should Have
-
A list of exam topics.
-
A description of the exam (length of time, type of questions,
materials they will need to supply)
-
The exam date and time.
Things You Should Do While Proctoring an Exam
-
Make sure that all students have the same amount of time
to work on the exam. This means fair start times and making sure all tests
are in when you call time.
-
Write the time remaining for the exam where the students
can see it.
-
Be available to answer questions. Write answers to general
questions on the board (mistakes on the exam questions for example).
-
Watch for students that may be cheating. If you think you
see this, document the incident. Do not confront the student about what
you suspect!!!!! You may ask a student to move to the front row. Let
the course instructor know about the incident or to deal with the situation
if they are present.
-
Make sure that you have all the copies of the test after
class.
Things You Should Have Prior to Grading
-
You should also have looked the exam over for errors prior
to giving it.
-
The answers to the exam
-
Grading criteria (you may have to generate these yourself
but make sure that the person who wrote the exam has a first pass at this).
Things You May Want Prior to Grading
-
A spreadsheet for recording student grades
Suggested Criteria Items
-
Correctness of the solution
-
Correctness of the procedures used to reach the solution
-
Creativity in approaching the problem
Suggested Procedures
-
Look over all the exams before starting. Get a feel for what
was answered well and what wasn’t. Look at what you think is the best answer
and what you think is the worst answer.
-
Make sure you have an uninterrupted block of time to look
these over.
-
Grade one problem at a time. (Grade everyone’s 1st
problem, then the 2nd, and so on.) When you finish with that
problem, look over the exams in reverse order to make sure you’ve graded
evenly. Try not to take a break between problems. Make sure you’ve graded
everyone’s exam for that problem before stopping. This will help reduce
inconsistency.
-
If the exam answers aren’t being made available to students,
make sure you write down what was incorrect about the problem. (It’s much
easier to make the answers available).
-
Make sure you explain why their approach was incorrect.
-
Double-check your math when you’re done.
-
It couldn’t hurt to generate a class average and standard
deviation.
Sample Criteria
-
Describe and diagram a 4-bit ripple carry adder. Show how
the following input will be evaluated by your adder: 101 + 110. (15 points)
-
Description of ripple carry adder (3 points)
-
2 points - correctly describes what a ripple carry adder
does
-
1 point - Mentions why it is called a "ripple-carry" adder.
-
Diagram of ripple carry adder ( 5 points)
-
4 points - correctly diagrammed AND/OR gates of adder
-
1 point - remembered input and outputs of adder.
-
Example diagrammed correctly (2 points)
-
1 point - returned correct answer
-
1 point - showed correct bits carried from one adder to the
next..
Notes made during grading
B - 2 points - logic is slightly wrong in an adder
but general implementation correct
B - 5 points - Only diagrammed one adder and 3 black
boxes, as long as adder is
correct. (Including input and output).
Problem / Short Answer Grading
Things the Students Should Have
-
A list of problems and page numbers or an assignment sheet
with the problems.
Things You Should Have Prior to Grading
-
You should look over the problems and generate your own answers
first.
-
The answers to the exam
-
Grading criteria
Things You May Want Prior to Grading
-
A spreadsheet for recording student grades
Suggested Criteria Items
-
Correctness of the solution
-
Correctness of the procedures used to reach the solution
-
Creativity in approaching the problem
Suggested Procedures
-
Make sure you understand what is being asked. Sometimes,
problems can be very ambiguous. If there are some different interpretations
of the problem resulting in different solutions or approaches, make sure
you understand these before you arbitrarily take off points for not meeting
your solution.
-
Grade one problem at a time. (Grade everyone’s 1st
problem, then the 2nd, and so on.) When you finish with that
problem, look over the assignments in reverse order to make sure you’ve
graded evenly. Try not to take a break between problems. Make sure you’ve
graded everyone’s assignment for that problem before stopping. This will
help reduce inconsistency.
-
If there are missing or unexplained steps in a proof or equation,
make sure you highlight these and ask the student to reflect on the logic.
-
If the answer is incorrect, check to see if it was an arithmetic
error (everyone hates being penalized for a stupid mistake - make sure
the logic is correct).
-
Give positive feedback for good solutions.
-
Make sure your math is correct when adding up the points.
Essay and Term Paper Grading
Things the Students Should Have
-
The Problem description and due date
-
A set of assignment requirements - including minimum length,
style guidelines, formatting, and topics. (Suggestion: Don’t accept handwritten
papers!!)
-
The source materials necessary to generate the Essay or Term
Paper
Things You Should Have Prior to Grading
-
You should also have looked the topics over.
-
You should make sure you have a fairly good understanding
of the class material.
-
Some general grading criteria. Because of the qualitative
nature of these types of assignments, you may want to develop some major
categories of points and assign students some % of the points per category
for meeting your criteria.
Things You May Want Prior to Grading
-
A spreadsheet for recording student grades
-
A large block of time to read and grade these.
Suggested Criteria Items
-
Accuracy of the logic used to argue the point of the paper
-
Documented support of the major points.
-
Correctness of the procedures used to reach the solution
-
Creativity in approaching the problem
-
Quality of the overall presentation
-
Quality of the overall paper layout and writing style.
Suggested Procedures
-
If the assignment is an essay, take a first pass at the first
5 and get a feel for what students are saying. If there are multiple topics,
divide up the assignments into stacks and evaluate each stack separately.
-
Don’t be too overly impressed with a good layout and diagrams.
Make sure that they set out a good argument or presentation and defended
it with well constructed statements, documented sources, or experimentation.
-
If there’s some ambiguity with what’s being said, make a
note of it and finish reading it first. The paper may resolve it later
on.
-
You should grade backwards and forwards to make sure you’re
applying the criteria evenly.
-
Grade first for content then style - unless the style is
glaringly poor. In the second case, you should make style corrections as
you go just so you don’t have to do it on a second pass.
-
Make sure your feedback asks questions about where the logic
failed or where you didn’t understand a point the paper was trying to make.
In some cases, you may be able to cite other papers or class readings that
they should have looked at. Make sure you encourage good argumentation.
-
When you’ve finished, it couldn’t hurt to look the whole
set over as a group.
-
Again, make sure your math is correct.
Sample Criteria
Final Project Grades
Team:
Members:
Grading:
|
Points
|
Total
|
Category |
| |
40
|
Final Report |
| |
5
|
Summary
|
| |
10
|
Project Process
|
| |
5
|
Support Documents
|
| |
15
|
Results of Implementation / Conclusion
|
| |
5
|
Future Work
|
| |
|
|
| |
20
|
Project Presentation |
| |
15
|
Content
|
| |
5
|
Delivery
|
| |
|
|
| |
20
|
Project Post
Mortem |
| |
5
|
Analysis of Project Schedule
|
| |
10
|
Analysis of Project Process
|
| |
5
|
Lessons Learned
|
| |
|
|
| |
10
|
Overall Management
and Organization of Project |
| |
|
|
| |
10
|
Overall Acceptance
Criteria of Project |
| |
|
|
| |
100
|
Total |
Comments:
Presentation Grading
Things the Students Should Have
-
Presentation Materials (slides, overheads, on-line presentations
and so on)
-
All team members should be present (assuming it’s a team
project).
Things You Should Have Prior to Grading
-
A Watch or Timer to keep track of presentation length.
-
A grading sheet for the group presenting.
Things You May Want Prior to Grading
-
Cards indicating the length of time remaining for the talk.
-
You may want to verify that all equipment is working properly
prior to class.
Suggested Criteria Items
-
Organization and coherence of the talk.
-
Quality of the content provided (does it address the assignment
parameters?)
-
Quality of the visual aids.
-
Logic used to argue the main points of the presentation.
-
The ability of the presenters to answer your questions and
those of the audience.
-
Creativity in the presentation.
Suggested Procedures
-
Make sure that all students have scheduled a day and time
to present a couple weeks in advance. This will help you to plan for those
days.
-
Set up guidelines in the assignments for the length of time
they will have to present and to answer questions.
-
If your students are inexperienced in giving presentations,
give them some suggestions.
-
Check to see if any presenters have any specific audio-visual
needs that you will need to address prior to the class.
-
If the presentation needs to be on time and finished on time,
make sure you have a watch handy and some way of reminding the presenter(s)
of the time remaining.
-
In a large room or auditorium, try to get microphones.
-
Make sure you have a grading sheet available.
-
Take notes during the talk on the grading sheet. This will
be feedback for the student.
Sample Grading Criteria
Final Presentation Grades
Team: _____________
Grading:
Please note that grading should be based mostly on the
quality of the presentation content and not on the content itself (this
will be addressed elsewhere in project grading and can be noted in Notes
for Grader). Notes about how the content should be graded will be made
in the sections provided below.
|
Points
|
Total
|
Category |
Description |
| |
50
|
Content |
|
| |
5
|
Opening Statement |
Presentation opening should discuss
goals, project objectives, motivation. |
| |
10
|
Project Process |
Discussion of project process
- research, implementation, development issues, etc. |
| |
15
|
Findings / Results |
Justification of project - did
team give good arguments or proof for how they addressed goals? |
| |
5
|
Closing Statement |
Talk closes with overview, conclusion,
and restatement of goals. |
| |
10
|
Discussion |
How well did team handle questions
from customer / instructor? |
| |
5
|
Overall Design |
How was flow of talk? |
| |
|
|
|
| |
20
|
Delivery |
|
| |
5
|
Clarity |
Is speaker(s) understandable? |
| |
10
|
Visuals |
Are visuals easy to read and understand? |
| |
5
|
Engagement |
Does speaker engage audience? |
| |
|
|
|
| |
5
|
Discretionary
Points |
These points
are assigned at speaker’s discretion for overall quality, effort, or novelty. |
| |
|
|
|
| |
75
|
Total |
|
Notes for Grader:
Comments for Group: