Skip to main content

​
Creating and Managing Rubrics

This guide explains how rubrics drive the hand-grading experience and how scores are computed. It also includes copy-pasteable YAML examples for each configuration option.
Rubrics can only be edited by instructors. When editing a rubric, you will be prompted to confirm before leaving the page if you have unsaved changes.

​
High-Level Overview

  • Rubric: Named collection of parts used for a specific review round (self-review, grading-review, meta-grading-review). The grading review is the main review that students will see.
  • Part: Logical section within a rubric (e.g., Code Quality, Functionality). Contains criteria. Parts can be configured for individual grading or assign-to-student modes for group submissions.
  • Criteria: A scoring rule block with:
    • is_additive: If true, checks add points up to total_points. If false, checks deduct points from total_points.
    • total_points: Max points for this criteria. It is not possible for a grader to assign more points to nested checks, even if you allow them to select that many points.
    • min_checks_per_submission / max_checks_per_submission: How many checks must/can be applied for the criteria.
    • One or more checks.
  • Check: The atom of feedback and scoring. A check can be global or an annotation applied to a file line or to an artifact. Checks can have:
    • points or selectable data.options with distinct labels/points
    • is_annotation + annotation_target (file or artifact), optional file/artifact to link
    • is_required: Must be applied by graders
    • is_comment_required: Comment is required when applying
    • max_annotations: Cap on times a check may be applied as annotations
    • student_visibility: always | if_applied | if_released | never
    • kpi_category: Optional link to repository analytics (commits, prs_opened, pr_review_comments, issues_opened, issues_closed, issue_comments)

​
Where This Shows Up in the Hand-Grader

  • The right-hand panel is the rubric sidebar. Global checks render as radio/checkbox options. Annotation checks are applied from the code view or artifact cards by right-clicking a line or selecting an artifact.
  • Each applied check creates a comment entry tied to a check with points and optional text.
  • For each criteria, the sidebar shows a running subtotal.

​
Artifact Rendering

When grading submissions, artifacts (non-code files) are displayed with appropriate rendering:
  • Markdown files (.md, .markdown): Rendered with full formatting including headers, lists, links, and code blocks
  • Plain text files (.txt): Displayed as formatted text for easy reading
  • Binary files: Available for download
This allows graders to provide feedback on documentation, reports, and other non-code deliverables directly within the grading interface. Annotation checks can target specific artifacts, enabling inline feedback on README files, design documents, and other text-based submissions.

​
How Points Are Computed

  • Per criteria:
    • If is_additive = true: criteria score = sum(applied check points) up to total_points.
    • If is_additive = false: criteria score = total_points - sum(applied deduction points) (floored at 0).
  • The rubric total is the sum across criteria.
  • Scores automatically recalculate when rubric check scores change, ensuring submission grade totals stay synchronized with the latest rubric configuration.

​
Note on Options vs Base Points

  • If a check has data.options, the selected option’s points replaces the check’s base points when applied.
  • Options must be 2+ (single options are disallowed by the editor and schema).

​
References Across Rubrics

  • Checks can reference checks from other rubrics to surface related feedback when grading. This is managed inline in the sidebar (preview mode).

​
YAML Schema Quick Reference

These examples conform to public/RubricSchema.json.

​
Minimal Rubric

name: Sample Rubric
parts:
  - name: Code Quality
    criteria:
      - name: Style and Clarity
        total_points: 10
        is_additive: false
        checks:
          - name: Missing Javadocs
            is_annotation: false
            is_required: false
            is_comment_required: false
            points: 2
          - name: Poor variable naming
            is_annotation: true
            annotation_target: file
            is_required: false
            is_comment_required: true
            max_annotations: 3
            points: 1

​
Additive vs Subtractive Criteria

name: Additive vs Subtractive
parts:
  - name: Functionality
    criteria:
      - name: Passing tests (additive)
        total_points: 20
        is_additive: true
        checks:
          - name: Public API works
            is_annotation: false
            is_required: false
            is_comment_required: false
            points: 5
          - name: Edge cases handled
            is_annotation: false
            is_required: false
            is_comment_required: false
            points: 5
      - name: Style deductions (subtractive)
        total_points: 10
        is_additive: false
        checks:
          - name: Magic numbers
            is_annotation: true
            annotation_target: file
            is_required: false
            is_comment_required: false
            points: 1
          - name: Redundant code
            is_annotation: true
            annotation_target: file
            is_required: false
            is_comment_required: true
            points: 2

​
Check with Options (Multiple Choice)

name: Options Example
parts:
  - name: API Correctness
    criteria:
      - name: HTTP response quality
        total_points: 10
        is_additive: true
        checks:
          - name: Response completeness
            is_annotation: false
            is_required: true
            is_comment_required: false
            points: 0
            data:
              options:
                - label: Complete and correct
                  points: 5
                - label: Mostly complete
                  points: 3
                - label: Incomplete
                  points: 1

​
Annotations (File vs Artifact) and Limits

name: Annotation Targets
parts:
  - name: Docs and Reports
    criteria:
      - name: README quality (deductions)
        total_points: 5
        is_additive: false
        checks:
          - name: Missing section
            is_annotation: true
            annotation_target: artifact
            artifact: README.md
            is_comment_required: true
            is_required: false
            max_annotations: 2
            points: 1
      - name: Code comments
        total_points: 5
        is_additive: false
        checks:
          - name: Unclear comment
            is_annotation: true
            annotation_target: file
            file: src/Main.java
            points: 1

​
Required Checks and Required Comments

name: Requireds
parts:
  - name: Process
    criteria:
      - name: Submission hygiene
        total_points: 5
        is_additive: true
        checks:
          - name: Compiles
            is_annotation: false
            is_required: true       # graders must apply
            is_comment_required: false
            points: 5
          - name: Explain deviation
            is_annotation: false
            is_required: false
            is_comment_required: true  # applying this check requires a comment
            points: 0

​
Min/Max Checks per Criteria

name: Min/Max Checks
parts:
  - name: Code Review
    criteria:
      - name: Choose exactly one pattern
        total_points: 5
        is_additive: true
        min_checks_per_submission: 1
        max_checks_per_submission: 1
        checks:
          - name: Builder pattern used
            is_annotation: false
            points: 5
          - name: Strategy pattern used
            is_annotation: false
            points: 5
      - name: Choose up to two strengths
        total_points: 4
        is_additive: true
        max_checks_per_submission: 2
        checks:
          - name: Test readability
            is_annotation: false
            points: 2
          - name: Modular design
            is_annotation: false
            points: 2

​
Student Visibility

name: Visibility
parts:
  - name: Feedback
    criteria:
      - name: Public notes
        total_points: 0
        is_additive: true
        checks:
          - name: General praise
            is_annotation: false
            points: 0
            student_visibility: always
      - name: Internal notes
        total_points: 0
        is_additive: true
        checks:
          - name: For staff only
            is_annotation: false
            points: 0
            student_visibility: never
      - name: Released only
        total_points: 0
        is_additive: true
        checks:
          - name: Visible when released
            is_annotation: false
            points: 0
            student_visibility: if_released
      - name: Only if applied
        total_points: 0
        is_additive: true
        checks:
          - name: Shown when applied
            is_annotation: false
            points: 0
            student_visibility: if_applied

​
Editing Rubrics

When editing rubrics, Pawtograder protects against accidental data loss. If you attempt to leave the rubric editor with unsaved changes, a confirmation dialog appears warning you that your changes will be lost. You can choose to stay and save, or discard the changes and leave. This protection applies when:
  • Navigating away via links within the application
  • Using browser back/forward buttons
  • Closing the browser tab or window
  • Refreshing the page
  • Using the mobile navigation dropdown
The warning only appears when you have actual unsaved changes in the rubric editor. After successfully saving, the warning clears automatically.

​
Keyboard shortcuts

Speed up rubric editing with keyboard shortcuts:
  • Ctrl/Cmd + S: Save rubric
  • Ctrl/Cmd + Z: Undo last change
  • Ctrl/Cmd + Shift + Z: Redo
  • Esc: Close current dialog
If you’re working on rubrics across multiple tabs, only the currently visible tab will manage the unsaved changes warning to avoid false alerts.
After reviews are released to students, grading comments cannot be edited or deleted (except by instructors — see Grading Comment Restrictions After Release). Plan your feedback carefully before releasing grades.

​
How This Renders in the Hand-Grader

  • Global checks appear as radio buttons (when max_checks_per_submission = 1) or checkboxes (otherwise).
  • Checks with data.options render a choice list. The selected option’s label is prefixed to the comment.
  • Annotation checks are applied via right-click on a code line or by selecting an artifact.
  • Checks linked to specific artifacts display a clickable link to navigate directly to that artifact.
  • For each criteria, the sidebar shows:
    • Additive: earned / total_points
    • Subtractive: remaining / total_points

​
Regrade Requests

Students can request regrades on rubric items where they believe the grading was incorrect. Regrade requests are automatically prevented on self-review rubric items, since those are completed by the students themselves.

​
Split-Rubric Group Grading

For group assignments, you can apply per-student adjustments when using split-rubric grading. This allows you to assign different scores or provide individualized feedback to group members while maintaining the efficiency of group grading workflows.

​
Editing Rubric Scores After Completion

TAs and instructors can edit or delete rubric scores even after completing a review. This allows for:
  • Correcting mistakes in grading
  • Adjusting scores based on regrade requests
  • Updating feedback after discussion with students
  • Modifying annotations or comments

​
How to Edit After Completion

  1. Navigate to the completed review
  2. Click on any rubric score or annotation
  3. Make your changes (modify points, update comments, or delete checks)
  4. Changes are saved automatically
  5. The review remains marked as complete unless you explicitly mark it incomplete
Editing completed reviews does not automatically notify students. If grades have been released, you may need to communicate changes to affected students.

​
Notes and Tips

  • Keep criteria focused and keep check names short; longer explanation should go in description.
  • Use max_annotations to prevent over-counting nitpicks.
  • Prefer options when the same check has graded tiers (e.g., Complete/Partial/Incomplete).
  • Use student_visibility to separate internal notes from student-facing feedback.
  • Grade override notes are now properly displayed in the grading interface.
  • Rubric scores are automatically recalculated when check scores change, ensuring the gradebook stays up-to-date.
  • Save frequently to avoid losing work, especially for complex rubrics.

​
Field Reference: Required vs Optional (with defaults)

Rubric
  • Mandatory: name, parts
  • Optional: description
Part
  • Mandatory: name, criteria
  • Optional: id, description, data
Criteria
  • Mandatory: name, checks
  • Optional: id, description, data, is_additive (default: false), total_points (default: 0), min_checks_per_submission, max_checks_per_submission
Check
  • Mandatory: name, is_annotation (boolean), is_required (boolean), is_comment_required (boolean), points (number; use 0 when the check relies on data.options)
  • Optional: id, description, file, artifact, annotation_target (file | artifact; default behavior in UI is file when omitted for annotations), max_annotations, data (see below), student_visibility (default: always)
Check data.options (for multiple-choice checks)
  • Mandatory per option: label, points
  • Optional per option: description
  • Notes: Must define at least two options (single-option checks are rejected by the editor). When options are present, the selected option’s points replace the base points for that check when applied.
Defaults and Behaviors Used by the Grader UI
  • Criteria without is_additive are treated as subtractive (deduction) criteria.
  • Criteria without total_points default to 0 (i.e., no contribution unless checks add points in additive mode).
  • Check student_visibility defaults to always.
  • Annotation annotation_target defaults to file in the UI if omitted.
  • min_checks_per_submission and max_checks_per_submission are optional; when not set, graders are not constrained by count.

​
Grading Comment Restrictions After Release

Once a review has been released to students, grading comments are restricted to maintain grading integrity:
  • Graders can only edit or delete their own comments after a review is released
  • Instructors retain full editing privileges and can modify any comment, even after release
  • This prevents graders from modifying feedback after students have seen it, while allowing instructors to make corrections if needed
This restriction applies to all rubric comments and annotations, ensuring that students see consistent feedback once grades are released.

​
Check References (Cross-Rubric Context)

​
What It Is

A check on one rubric can reference a check on another rubric. During grading, any applied feedback (comments/points) from the referenced check(s) is shown inline under the current check as “Related Feedback from Other Reviews”. This gives graders context from, e.g., self-review or prior review rounds.

​
Scoring and Visibility

  • References are informational only. They do not contribute points to the current rubric’s criteria and do not alter the score computation.
  • Referenced feedback is surfaced to graders in grading mode. Student visibility continues to follow each original check’s own student_visibility and release state within its source review.

​
How to Configure

  1. Navigate to the rubric preview/editor for an assignment.
  2. For the check you want to augment, click “Add Reference”.
  3. Search/select a check from other rubrics (the current rubric’s checks are excluded).
  4. Save. The relationship is stored so that, when grading, the referenced feedback appears under the referencing check.

​
Good Use Cases

  • Show a student’s self-review evidence next to the corresponding grading check.
  • Pull in meta-grading notes when doing final pass reviews.

​
Individual and Assign-to-Student Grading Modes

For group submissions, rubric parts can be configured with special grading modes to assess individual contributions:

​
Individual Grading Mode

When a rubric part has is_individual_grading: true, that part is graded separately for each group member. The grading interface displays the part once per student, and graders apply checks individually for each team member. Use cases:
  • Individual code quality assessments in group projects
  • Personal reflection questions
  • Individual contribution evaluations
Configuration: 
name: Group Project Rubric
parts:
  - name: Individual Code Quality
    is_individual_grading: true
    criteria:
      - name: Code style
        total_points: 10
        is_additive: false
        checks:
          - name: Style violations
            is_annotation: true
            annotation_target: file
            points: 1
When grading, you’ll see this part repeated for each group member with a student name badge. Annotations and checks must be applied separately for each student.

​
Assign-to-Student Mode

When a rubric part has is_assign_to_student: true, the grader selects which student the part applies to (or skips it if not applicable). Use cases:
  • Role-specific responsibilities (e.g., “Team Lead” duties)
  • Optional bonus sections that only some students completed
  • Targeted feedback for specific contributions
Configuration: 
name: Group Project Rubric
parts:
  - name: Team Lead Responsibilities
    is_assign_to_student: true
    criteria:
      - name: Leadership
        total_points: 5
        is_additive: true
        checks:
          - name: Effective coordination
            is_annotation: false
            points: 5
When grading, you’ll see a dropdown to select which student this part applies to, or choose “Skip” if it doesn’t apply to anyone.

​
Scoring with Individual/Assign-to-Student Parts

When a rubric includes individual or assign-to-student parts:
  • Shared score: Points from regular (whole-group) rubric parts, autograder, and tweak
  • Individual score: Points from individual-grading and assign-to-student parts for each student
  • Total per student: Shared + individual, capped to assignment maximum if configured
The grading interface displays a “Scores by student” breakdown showing shared, individual, and total scores for each group member.

​
Important Notes

  • A part cannot be both is_individual_grading and is_assign_to_student (mutually exclusive)
  • Individual grading parts require target_student_profile_id on all rubric-linked comments
  • Assign-to-student parts are skipped in completion validation when not assigned to any student
  • Students only see their own individual scores, not their teammates’ scores