Using Rubric Criteria and Levels to Ensure Accuracy in Peer Assessment

"I am a big believer in rubrics; they harmonize everyone's expectations on an assignment. Kritik's features include pre-built rubrics for any type of activity you plan on assigning, so I use those and tweak them to ensure that they apply to my class' learning objectives." - Dr. Jeff Boggs

Whether you’ve used peer assessment in your courses or not, there is a daunting fear about the accuracy and fairness of peer-given marks. But by building a comprehensive rubric, you can guide your students when doing peer-to-peer evaluations that can also be used as a learning opportunity.

Typically, students find most rubrics vague and unhelpful due to the criteria for each level using terms that may mean different things to different students, such as good, excellent, outstanding etc. For instructors, it can be time-consuming and difficult to create rubrics from scratch that are detailed enough for students to understand what is expected of them.

With multiple perspectives of the students who need to read the rubric and the instructor, who perceives things differently from students, it can be a struggle to write a rubric that satisfies all people involved. In this article, we will be talking about some common concerns that we’ve heard from instructors about building rubrics with some guidelines that can help achieve this.

1. How many criteria should the rubric have?

There is no universal optimal number of criteria, but “the general consensus” is that less is more[1]. Overall, the number of criteria should be related to the learning outcome(s) the assignment assesses[2]. The project's complexity will also factor in choosing the number of criteria that should be included in your rubrics.

Best Practice: Criteria

Research suggests that once rubrics get too lengthy, it becomes difficult for students to understand the main focus of the assignment and the key skills they need to achieve[3]. Thus, Quinlan suggests using 3-4 to start with[4], and recommend a maximum of 7 criteria[5].

Watch how easy it is to create your rubrics for peer assessment on Kritik!

2. How many levels should the rubrics have?

Similarly to the number of criteria, rubric levels should follow the “less is more” concept. There need to be enough levels where the students can understand the expectations from the instructor, but not too many where students can understand the differences between the levels.

‍Best Practice: Levels

Like criteria, the optimal number will vary, but a high-quality rubric should typically consist of 3-5 levels[6]. A minimum of three is recommended so there are enough levels to represent adequate work, inadequate work, and an exceptional level to motivate students to go above and beyond[7]. On the other hand, no more than five is recommended because having too many performance levels can make it harder to distinguish the differences between them[8].

"They see the rubric when they submit their assignments, and they use the rubrics to evaluate each other, so it trains them to respond to objectives. It also manages a student's expectation of what they need to do to achieve the grade they want" - Dr. Erin Panda

3. Do I need to narrow down the rubric criteria?

To help determine the number of criteria to include, MIT[9] suggests considering the following:

  • What is the learning goal of the activity?
  • How will students demonstrate that they have achieved the learning goal(s)?
  • What knowledge and skills are required to succeed?
  • What characteristics should the final product have?

After answering these questions and using them to develop a list of potential criteria, the next step is prioritizing the most important traits. You will need to eliminate unnecessary criteria and club together similar ones to shorten the rubric to its essentials.

4. How can I ensure grading accuracy

By using specific terms in your criteria, it allows students to understand exactly what the instructor is looking for and leaves no ambiguity when they are evaluating each others’ work. 

For example, if the criterion were simply "include three properly cited sources," there would be three relevant distinctions:

The same thing would apply to more subjective criteria, such as creativity. Relevant, distinctive language can be used, such as:

Watch how instructors use Kritik to design their rubrics for peer assessment.

5. Are my rubrics aligned with the learning objectives?

What are your learning objectives for the assignment you give your students? Your criteria need to include sections that support your learning objectives. For example, in reflection activities, a section is needed where students are informed on how many course materials they need to complete the assessment successfully.

💡Pro Tip: The descriptors should clearly articulate the expectations for each performance level for a given criterion. Each row should represent meaningful differences in performance across the performance levels for a given criterion[10].

Design Rubrics for Peer Assessment with Kritik!

Building rubrics for peer evaluations with no room for ambiguity while meeting all the learning goals of the course can be daunting. With Kritik, instructors get a large bank of customizable rubrics that have been created by instructional designers that can be used across disciplines.

Schedule a demo with Kritik today to create rubrics that can guide students to doing peer evaluations more accurately.

References

[1] Secolsky, C., & Denison, D. B. (2018). Handbook on measurement, assessment, and evaluation in Higher Education. Routledge. 

[2]Suskie, L. (2017). Rubric Development. In Handbook on Measurement, Assessment, and Evaluation in Higher Education (pp. 549–550). 

[3]Lane, S. (2010). Performance assessment: The state of the art. (SCOPE Student Performance Assessment Series). Stanford, CA: Stanford University, Stanford Center for Opportunity Policy in Education

[4]Quinlan, A. M. (2012). A complete guide to rubrics: Assessment made easy for teachers of K–college (2nd ed.). Lanham, MD: Rowman & Littlefield.

[5]Stevens, D. D., & Levi, A. J. (2012). Introduction to rubrics: An assessment tool to save grading time, convey effective feedback and promote student learning. Stylus Publishing. 

[6] [7] [8] Brookhart, S. M. (2018, April 10). Appropriate criteria: Key to effective rubrics. Frontiers. Retrieved February 7, 2022, from https://www.frontiersin.org/articles/10.3389/feduc.2018.00022/full 

[9] Suskie, L. (2009). Using a scoring guide or rubric to plan and evaluate an assessment. In Assessing student learning: A common sense guide (2nd edition, pp. 137-154). Jossey-Bass.

[10] Massachusetts Institute of Technology. (n.d.). How to use Rubrics. How to Use Rubrics | Teaching + Learning Lab. Retrieved December 13, 2022, from https://tll.mit.edu/teaching-resources/assess-learning/how-to-use-rubrics/  

Emily Masters
Education Researcher

Heading