Calibrated Peer Review
If you've ever spent 20+ hours grading papers or assignments, then you know how powerful automated online grading software is. For online grading software to be effective, it must have clear grading standards defined, specified solutions stated and accurate results. Traditional grading systems encompass simple software to assess True/False or Multiple Choice Questions, however this is not feasible for short answer or written assignments where one must assess a students' deep understanding of course concepts.
Educators' dedication and efforts are widely appreciated and can be seen through how they conduct their courses. According to MacLean's News outlet, University professors work an average of 48 hours a week completing both teaching and non-teaching duties.
As many classrooms are being pushed to incorporate educational technology for remote teaching, why can't the same be applied for automated grading systems? We understand automated online grading software can most certainly free up more teaching time for professors, however, will it come at the cost of sacrificing fair grades for more biased or inaccurate grades?
More Time Saved, Less Time Wasted
To solve these challenges, Kritik is excited to announce our new calibration feature. This powerful tool is built to automatically keep our online grading software more precise while raising awareness among students regarding grading expectations for assignments and activities.
If you have already implemented Kritik as your e-learning platform, rolling out the calibration feature will be very easy. Currently, Kritik's grading system revolves around peer grading, where students anonymously evaluate their peer's submissions through rubric-based assessments. Now with calibrated peer review, students who grade closely to the professors preferences will have a higher impact when making evaluations on other submissions.
How does it work?
A professor will select, review and evaluate three creations of a previously finalized Kritik activity. Students will then evaluate the same three creations and have their evaluations provided for those same creations compared with the evaluations provided by the professor.
Students who have marked in line with how the professor did will get a higher grading score for the calibration activity. The higher the grading score the student achieves, the greater the impact they will have on future activities when assessing other students ' creations.
The aim is to identify strong peer evaluators who will prove to be effective for peer grading. Students who mark closely or exactly in line with the professor will demonstrate how strong their grading skills really are. In return, those very students will be rewarded with a higher grading score. Likewise, students who marked poorly or with little effort will be penalized with a lower grading score.
Tying it Together
The creation score is an overall weighted average of the evaluations that a student received on their submission. In this case, the weight is the peer evaluators grading score. Students who increase their total grading points from the calibration activity will have a higher weight as they evaluate their peer submissions. This can be proven valuable for essay marking, lab marking and etc.
- Improve Grading Accuracy: You are guaranteed less inflated grades through calibrated peer review. Administering a calibration activity 1-2 times over the whole term is more than enough.
- Identify Key Evaluators: Students who grade closely to you indicate how well they are absorbing the context of the assignment requirements and rubric criteria. You can count on these students alike to the TA's to provide effective assessments and feedback that will further support their peers in the course.
- Discovering Potential: Kritik allows students to develop their evaluative abilities as they practice to provide consistent feedback for every activity. For large class sizes, students who are well performing and achieve a high Evaluation Score can be a great indication of their evaluation ability. It provides a perfect opportunity for professors to identify students that can be recommended as a TA.
- Gamification: To keep students motivated as they complete assignments, the Kritik scoring system allows students to keep track of their improvement throughout the term. Obtaining a high grading score showcases how well students are comprehending and applying course content.
- Offer more assignments: With the calibration feature in use, you can spend less time revising the grades of the students and spending more time creating Kritik assignments. With at least 6-7 Kritik activities by the end of the term, you can expect to see an exponential increase in the student's Evaluation score.
Get Started Today
Brownell, C. (2018, May 17). Canadian university professors spend roughly half of their time on teaching. Retrieved from https://www.macleans.ca/education/canadian-university-professors-spend-equal-amounts-of-time-on-teaching-and-non-teaching-work/