Creating and Managing Rubrics

This guide explains how rubrics drive the hand-grading experience and how scores are computed. It also includes copy-pasteable YAML examples for each configuration option.

Rubrics can only be edited by instructors. When editing a rubric, you will be prompted to confirm before leaving the page if you have unsaved changes.

High-Level Overview

Rubric: Named collection of parts used for a specific review round (self-review, grading-review, meta-grading-review). The grading review is the main review that students will see.
Part: Logical section within a rubric (e.g., Code Quality, Functionality). Contains criteria. Parts can be configured for individual grading or assign-to-student modes for group submissions.
Criteria: A scoring rule block with:
- is_additive and is_deduction_only: together these pick one of three scoring modes (see How Points Are Computed). Both keys may be present; is_deduction_only: true selects “Penalty only” whatever is_additive says, and is_additive is read only when is_deduction_only is false or absent. The editor writes one mode at a time.
- total_points: Max points for this criteria. It is not possible for a grader to assign more points to nested checks, even if you allow them to select that many points.
- min_checks_per_submission / max_checks_per_submission: How many checks must/can be applied for the criteria.
- One or more checks.
Check: The atom of feedback and scoring. A check can be global or an annotation applied to a file line or to an artifact. Checks can have:
- points or selectable data.options with distinct labels/points
- is_annotation + annotation_target (file or artifact), optional file/artifact to link
- is_required: Must be applied by graders
- is_comment_required: Comment is required when applying
- max_annotations: Cap on times a check may be applied as annotations
- student_visibility: always | if_applied | if_released | never
- kpi_category: Optional link to repository analytics (commits, prs_opened, pr_review_comments, issues_opened, issues_closed, issue_comments)

Where This Shows Up in the Hand-Grader

The right-hand panel is the rubric sidebar. Global checks render as radio/checkbox options. Annotation checks are applied from the code view or artifact cards by right-clicking a line or selecting an artifact.
Each applied check creates a comment entry tied to a check with points and optional text.
For each criteria, the sidebar shows a running subtotal.

Artifact Rendering

When grading submissions, artifacts (non-code files) are displayed with appropriate rendering:

Markdown files (.md, .markdown): Rendered with full formatting including headers, lists, links, and code blocks
Plain text files (.txt): Displayed as formatted text for easy reading
Binary files: Available for download

This allows graders to provide feedback on documentation, reports, and other non-code deliverables directly within the grading interface. Annotation checks can target specific artifacts, enabling inline feedback on README files, design documents, and other text-based submissions.

How Points Are Computed

Each criteria uses one of three scoring modes. The rubric editor names them “Award per check”, “Deduct from total”, and “Penalty only”; in YAML they are two booleans. is_deduction_only: true wins outright — is_additive is consulted only when is_deduction_only is false or omitted.

Mode in the editor	YAML	Criteria score
Award per check	`is_additive: true`	sum(applied check points), capped at `total_points`
Deduct from total	`is_additive: false`	`total_points - sum(applied deduction points)`, floored at 0
Penalty only	`is_deduction_only: true`	`-sum(applied check points)`, with the penalty capped at `total_points` — starts at zero and only ever subtracts

“Penalty only” is the mode that is easy to miss. Unlike “Deduct from total” it does not begin from total_points, so a criteria with no checks applied contributes 0 rather than full marks, and its best possible contribution is 0. Here total_points acts as a floor on the penalty rather than a starting balance. Use this mode for penalties that should not inflate the rubric’s maximum, such as a late or academic-integrity deduction.

Both booleans are optional in YAML and default to false, which yields “Deduct from total”. If you want “Award per check” you must say is_additive: true explicitly.

The rubric total is the sum across criteria.
Scores automatically recalculate when rubric check scores change, ensuring submission grade totals stay synchronized with the latest rubric configuration.

Note on Options vs Base Points

If a check has data.options, the selected option’s points replaces the check’s base points when applied.
Options must be 2+ (single options are disallowed by the editor and schema).

References Across Rubrics

Checks can reference checks from other rubrics to surface related feedback when grading. This is managed inline in the sidebar (preview mode).

YAML Schema Quick Reference

These examples conform to public/RubricSchema.json.

Minimal Rubric

name: Sample Rubric
parts:
  - name: Code Quality
    criteria:
      - name: Style and Clarity
        total_points: 10
        is_additive: false
        checks:
          - name: Missing Javadocs
            is_annotation: false
            is_required: false
            is_comment_required: false
            points: 2
          - name: Poor variable naming
            is_annotation: true
            annotation_target: file
            is_required: false
            is_comment_required: true
            max_annotations: 3
            points: 1

Additive vs Subtractive Criteria

name: Additive vs Subtractive
parts:
  - name: Functionality
    criteria:
      - name: Passing tests (additive)
        total_points: 20
        is_additive: true
        checks:
          - name: Public API works
            is_annotation: false
            is_required: false
            is_comment_required: false
            points: 5
          - name: Edge cases handled
            is_annotation: false
            is_required: false
            is_comment_required: false
            points: 5
      - name: Style deductions (subtractive)
        total_points: 10
        is_additive: false
        checks:
          - name: Magic numbers
            is_annotation: true
            annotation_target: file
            is_required: false
            is_comment_required: false
            points: 1
          - name: Redundant code
            is_annotation: true
            annotation_target: file
            is_required: false
            is_comment_required: true
            points: 2

Check with Options (Multiple Choice)

name: Options Example
parts:
  - name: API Correctness
    criteria:
      - name: HTTP response quality
        total_points: 10
        is_additive: true
        checks:
          - name: Response completeness
            is_annotation: false
            is_required: true
            is_comment_required: false
            points: 0
            data:
              options:
                - label: Complete and correct
                  points: 5
                - label: Mostly complete
                  points: 3
                - label: Incomplete
                  points: 1

Annotations (File vs Artifact) and Limits

name: Annotation Targets
parts:
  - name: Docs and Reports
    criteria:
      - name: README quality (deductions)
        total_points: 5
        is_additive: false
        checks:
          - name: Missing section
            is_annotation: true
            annotation_target: artifact
            artifact: README.md
            is_comment_required: true
            is_required: false
            max_annotations: 2
            points: 1
      - name: Code comments
        total_points: 5
        is_additive: false
        checks:
          - name: Unclear comment
            is_annotation: true
            annotation_target: file
            file: src/Main.java
            points: 1

Required Checks and Required Comments

name: Requireds
parts:
  - name: Process
    criteria:
      - name: Submission hygiene
        total_points: 5
        is_additive: true
        checks:
          - name: Compiles
            is_annotation: false
            is_required: true       # graders must apply
            is_comment_required: false
            points: 5
          - name: Explain deviation
            is_annotation: false
            is_required: false
            is_comment_required: true  # applying this check requires a comment
            points: 0

Min/Max Checks per Criteria

name: Min/Max Checks
parts:
  - name: Code Review
    criteria:
      - name: Choose exactly one pattern
        total_points: 5
        is_additive: true
        min_checks_per_submission: 1
        max_checks_per_submission: 1
        checks:
          - name: Builder pattern used
            is_annotation: false
            points: 5
          - name: Strategy pattern used
            is_annotation: false
            points: 5
      - name: Choose up to two strengths
        total_points: 4
        is_additive: true
        max_checks_per_submission: 2
        checks:
          - name: Test readability
            is_annotation: false
            points: 2
          - name: Modular design
            is_annotation: false
            points: 2

Student Visibility

name: Visibility
parts:
  - name: Feedback
    criteria:
      - name: Public notes
        total_points: 0
        is_additive: true
        checks:
          - name: General praise
            is_annotation: false
            points: 0
            student_visibility: always
      - name: Internal notes
        total_points: 0
        is_additive: true
        checks:
          - name: For staff only
            is_annotation: false
            points: 0
            student_visibility: never
      - name: Released only
        total_points: 0
        is_additive: true
        checks:
          - name: Visible when released
            is_annotation: false
            points: 0
            student_visibility: if_released
      - name: Only if applied
        total_points: 0
        is_additive: true
        checks:
          - name: Shown when applied
            is_annotation: false
            points: 0
            student_visibility: if_applied

Editing Rubrics

When editing rubrics, Pawtograder protects against accidental data loss. If you attempt to leave the rubric editor with unsaved changes, a confirmation dialog appears warning you that your changes will be lost. You can choose to stay and save, or discard the changes and leave. This protection applies when:

Navigating away via links within the application
Using browser back/forward buttons
Closing the browser tab or window
Refreshing the page
Using the mobile navigation dropdown

The warning only appears when you have actual unsaved changes in the rubric editor. After successfully saving, the warning clears automatically.

Keyboard shortcuts

Speed up rubric editing with keyboard shortcuts:

Ctrl/Cmd + S: Save rubric
Ctrl/Cmd + Z: Undo last change
Ctrl/Cmd + Shift + Z: Redo
Esc: Close current dialog

If you’re working on rubrics across multiple tabs, only the currently visible tab will manage the unsaved changes warning to avoid false alerts.

After reviews are released to students, grading comments cannot be edited or deleted (except by instructors — see Grading Comment Restrictions After Release). Plan your feedback carefully before releasing grades.

How This Renders in the Hand-Grader

Global checks appear as radio buttons (when max_checks_per_submission = 1) or checkboxes (otherwise).
Checks with data.options render a choice list. The selected option’s label is prefixed to the comment.
Annotation checks are applied via right-click on a code line or by selecting an artifact.
Checks linked to specific artifacts display a clickable link to navigate directly to that artifact.
For each criteria, the sidebar shows:
- Additive: earned / total_points
- Subtractive: remaining / total_points

Regrade Requests

Students can request regrades on rubric items where they believe the grading was incorrect. Regrade requests are automatically prevented on self-review rubric items, since those are completed by the students themselves.

Split-Rubric Group Grading

For group assignments, you can apply per-student adjustments when using split-rubric grading. This allows you to assign different scores or provide individualized feedback to group members while maintaining the efficiency of group grading workflows.

Editing Rubric Scores After Completion

TAs and instructors can edit or delete rubric scores even after completing a review. This allows for:

Correcting mistakes in grading
Adjusting scores based on regrade requests
Updating feedback after discussion with students
Modifying annotations or comments

How to Edit After Completion

Navigate to the completed review
Click on any rubric score or annotation
Make your changes (modify points, update comments, or delete checks)
Changes are saved automatically
The review remains marked as complete unless you explicitly mark it incomplete

Editing completed reviews does not automatically notify students. If grades have been released, you may need to communicate changes to affected students.

Notes and Tips

Keep criteria focused and keep check names short; longer explanation should go in description.
Use max_annotations to prevent over-counting nitpicks.
Prefer options when the same check has graded tiers (e.g., Complete/Partial/Incomplete).
Use student_visibility to separate internal notes from student-facing feedback.
Grade override notes are now properly displayed in the grading interface.
Rubric scores are automatically recalculated when check scores change, ensuring the gradebook stays up-to-date.
Save frequently to avoid losing work, especially for complex rubrics.

Field Reference: Required vs Optional (with defaults)

Rubric

Mandatory: name, parts
Optional: description

Part

Mandatory: name, criteria
Optional: id, description, data

Criteria

Mandatory: name, checks
Optional: id, description, data, is_additive (default: false), total_points (default: 0), min_checks_per_submission, max_checks_per_submission

Check

Mandatory: name, is_annotation (boolean), is_required (boolean), is_comment_required (boolean), points (number; use 0 when the check relies on data.options)
Optional: id, description, file, artifact, annotation_target (file | artifact; default behavior in UI is file when omitted for annotations), max_annotations, data (see below), student_visibility (default: always)

Check data.options (for multiple-choice checks)

Mandatory per option: label, points
Optional per option: description
Notes: Must define at least two options (single-option checks are rejected by the editor). When options are present, the selected option’s points replace the base points for that check when applied.

Defaults and Behaviors Used by the Grader UI

Criteria without is_additive are treated as subtractive (deduction) criteria.
Criteria without total_points default to 0 (i.e., no contribution unless checks add points in additive mode).
Check student_visibility defaults to always.
Annotation annotation_target defaults to file in the UI if omitted.
min_checks_per_submission and max_checks_per_submission are optional; when not set, graders are not constrained by count.

Grading Comment Restrictions After Release

Once a review has been released to students, grading comments are restricted to maintain grading integrity:

Graders can only edit or delete their own comments after a review is released
Instructors retain full editing privileges and can modify any comment, even after release
This prevents graders from modifying feedback after students have seen it, while allowing instructors to make corrections if needed

This restriction applies to all rubric comments and annotations, ensuring that students see consistent feedback once grades are released.

Check References (Cross-Rubric Context)

What It Is

A check on one rubric can reference a check on another rubric. During grading, any applied feedback (comments/points) from the referenced check(s) is shown inline under the current check as “Related Feedback from Other Reviews”. This gives graders context from, e.g., self-review or prior review rounds.

Scoring and Visibility

References are informational only. They do not contribute points to the current rubric’s criteria and do not alter the score computation.
Referenced feedback is surfaced to graders in grading mode. Student visibility continues to follow each original check’s own student_visibility and release state within its source review.

How to Configure

Navigate to the rubric preview/editor for an assignment.
For the check you want to augment, click “Add Reference”.
Search/select a check from other rubrics (the current rubric’s checks are excluded).
Save. The relationship is stored so that, when grading, the referenced feedback appears under the referencing check.

Good Use Cases

Show a student’s self-review evidence next to the corresponding grading check.
Pull in meta-grading notes when doing final pass reviews.

Individual and Assign-to-Student Grading Modes

For group submissions, rubric parts can be configured with special grading modes to assess individual contributions:

Individual Grading Mode

When a rubric part has is_individual_grading: true, that part is graded separately for each group member. The grading interface displays the part once per student, and graders apply checks individually for each team member. Use cases:

Individual code quality assessments in group projects
Personal reflection questions
Individual contribution evaluations

Configuration:

name: Group Project Rubric
parts:
  - name: Individual Code Quality
    is_individual_grading: true
    criteria:
      - name: Code style
        total_points: 10
        is_additive: false
        checks:
          - name: Style violations
            is_annotation: true
            annotation_target: file
            points: 1

When grading, you’ll see this part repeated for each group member with a student name badge. Annotations and checks must be applied separately for each student.

Assign-to-Student Mode

When a rubric part has is_assign_to_student: true, the grader selects which student the part applies to (or skips it if not applicable). Use cases:

Role-specific responsibilities (e.g., “Team Lead” duties)
Optional bonus sections that only some students completed
Targeted feedback for specific contributions

Configuration:

name: Group Project Rubric
parts:
  - name: Team Lead Responsibilities
    is_assign_to_student: true
    criteria:
      - name: Leadership
        total_points: 5
        is_additive: true
        checks:
          - name: Effective coordination
            is_annotation: false
            points: 5

When grading, you’ll see a dropdown to select which student this part applies to, or choose “Skip” if it doesn’t apply to anyone.

Scoring with Individual/Assign-to-Student Parts

When a rubric includes individual or assign-to-student parts:

Shared score: Points from regular (whole-group) rubric parts, autograder, and tweak
Individual score: Points from individual-grading and assign-to-student parts for each student
Total per student: Shared + individual, capped to assignment maximum if configured

The grading interface displays a “Scores by student” breakdown showing shared, individual, and total scores for each group member.

Important Notes

A part cannot be both is_individual_grading and is_assign_to_student (mutually exclusive)
Individual grading parts require target_student_profile_id on all rubric-linked comments
Assign-to-student parts are skipped in completion validation when not assigned to any student
Students only see their own individual scores, not their teammates’ scores

​Creating and Managing Rubrics

​High-Level Overview

​Where This Shows Up in the Hand-Grader

​Artifact Rendering

​How Points Are Computed

​Note on Options vs Base Points

​References Across Rubrics

​YAML Schema Quick Reference

​Minimal Rubric

​Additive vs Subtractive Criteria

​Check with Options (Multiple Choice)

​Annotations (File vs Artifact) and Limits

​Required Checks and Required Comments

​Min/Max Checks per Criteria

​Student Visibility

​Editing Rubrics

​Keyboard shortcuts

​How This Renders in the Hand-Grader

​Regrade Requests

​Split-Rubric Group Grading

​Editing Rubric Scores After Completion

​How to Edit After Completion

​Notes and Tips

​Field Reference: Required vs Optional (with defaults)

​Grading Comment Restrictions After Release

​Check References (Cross-Rubric Context)

​What It Is

​Scoring and Visibility

​How to Configure

​Good Use Cases

​Individual and Assign-to-Student Grading Modes

​Individual Grading Mode

​Assign-to-Student Mode

​Scoring with Individual/Assign-to-Student Parts

​Important Notes

Creating and Managing Rubrics

High-Level Overview

Where This Shows Up in the Hand-Grader

Artifact Rendering

How Points Are Computed

Note on Options vs Base Points

References Across Rubrics

YAML Schema Quick Reference

Minimal Rubric

Additive vs Subtractive Criteria

Check with Options (Multiple Choice)

Annotations (File vs Artifact) and Limits

Required Checks and Required Comments

Min/Max Checks per Criteria

Student Visibility

Editing Rubrics

Keyboard shortcuts

How This Renders in the Hand-Grader

Regrade Requests

Split-Rubric Group Grading

Editing Rubric Scores After Completion

How to Edit After Completion

Notes and Tips

Field Reference: Required vs Optional (with defaults)

Grading Comment Restrictions After Release

Check References (Cross-Rubric Context)

What It Is

Scoring and Visibility

How to Configure

Good Use Cases

Individual and Assign-to-Student Grading Modes

Individual Grading Mode

Assign-to-Student Mode

Scoring with Individual/Assign-to-Student Parts

Important Notes