​

AI assistance for helping students

​

Overview

• Help requests: Get AI help when responding to student questions in office hours
• Discussion threads: Analyze student questions and draft responses
• Test failures: Understand why students are failing specific tests
• Build errors: Diagnose compilation and build issues
• Test insights: Analyze common error patterns across multiple submissions

​

How It Works

• Context about the student’s question or issue
• Links to relevant assignment details and handout URLs
• Suggested MCP tools for fetching additional data
• Guidance on how to structure your analysis

​

Setting Up AI Assistance

​

Prerequisites

• You must be an instructor or grader in the course
• Claude Desktop (recommended) or another MCP-compatible AI assistant installed

npx @anthropic-ai/mcp-proxy@latest

{
  "mcpServers": {
    "pawtograder": {
      "command": "npx",
      "args": ["-y", "@anthropic-ai/mcp-proxy@latest", "YOUR_PAWTOGRADER_URL/functions/v1/mcp-server"],
      "env": {
        "MCP_HEADERS": "{\"Authorization\": \"Bearer YOUR_TOKEN_HERE\"}"
      }
    }
  }
}

​

Using AI Assistance

​

In Help Requests

1. Click the AI Help button (robot icon) in the help request chat
2. The system copies a pre-formatted prompt to your clipboard
3. Paste the prompt into Claude Desktop
4. Claude will automatically fetch:
   • The student’s question and conversation history
   • Their latest submission with test results
   • The assignment specification and handout URL
   • Related discussion threads

• Diagnosis with evidence: Specific code snippets and test outputs showing the issue
• Two draft responses: Socratic (exploratory) and Direct (efficient) versions
• Verification checklist: Items to confirm before sending

​

In Discussion Threads

1. Click the AI Help icon in the thread actions
2. Paste the copied prompt into Claude
3. Claude fetches the thread, replies, and related context
4. Review the suggested response and adapt as needed

​

For Test Failures and Build Errors

1. Click the AI Help icon next to a failing test or build error
2. Paste the copied prompt into your AI assistant
3. Claude analyzes:
   • The test output and error messages (or build output)
   • The student’s source code
   • The assignment specification
   • The expected behavior

​

For Common Error Patterns

1. Click AI Analyze on an error group
2. Claude examines submissions from multiple students
3. Determines whether it’s a student error or an assignment/test issue
4. Drafts a discussion post you can share with the class

​

Available MCP Tools

​

Context Tools

• get_help_request — Fetch help request with conversation history
• get_discussion_thread — Fetch discussion thread with replies
• get_assignment — Get assignment details, handout URL, and rubric

​

Submission Tools

• get_submission — Get submission metadata and summary
• list_submission_files — See what files the student submitted
• get_submission_files — Fetch specific source files using glob patterns
• list_submission_tests — See all test results (pass/fail, scores)
• get_test_output — Get detailed output for specific tests
• get_submission_build_output — Get compilation/build errors

​

Repository Tools

• list_grader_files — List files in the grader/solution repository
• get_grader_files — Fetch grader files (tests, mutants, solutions)
• list_handout_files — List files in the template/handout repository
• get_handout_files — Fetch handout/starter code files

​

Search Tools

• search_help_requests — Find related help requests
• search_discussion_threads — Find related discussion threads
• get_submissions_for_student — Get all submissions for a student

​

Handout URLs

1. Navigate to the assignment creation form
2. Fill in the Handout URL field with a link to your assignment instructions
3. The URL will be provided to AI assistants when analyzing student issues

​

Privacy and Security

• Access control: Only instructors and graders can use AI assistance
• Class-level authorization: You can only access data from classes where you have instructor or grader role
• Privacy protection: The system never exposes user email addresses or private profile flags
• Token-based authentication: API tokens are long-lived JWTs that can be revoked at any time
• Audit trail: Token usage is tracked with last-used timestamps

​

Providing Feedback

• Thumbs up/down: Rate whether the AI analysis was helpful
• Optional comment: Provide details about what worked or didn’t work

​

Managing API Tokens

​

Viewing Your Tokens

1. Click your profile menu
2. Select Settings → API Tokens
3. View all your tokens with their status, creation date, and last used date

​

Creating a Token

1. Click Create New Token
2. Enter a descriptive name (e.g., “Claude Desktop”, “VS Code”)
3. Copy the token immediately — it will only be shown once

​

Revoking a Token

1. Find the token in your token list
2. Click the trash icon
3. Confirm revocation

​

Best Practices

​

When to Use AI Assistance

• Complex debugging: When a student’s error is difficult to diagnose or spans multiple layers
• Pattern recognition: When multiple students have similar issues
• Time-saving: When you need to draft a detailed, well-structured response quickly
• Learning: When you want to see different approaches to explaining a concept

​

When to Skip AI Assistance

• Simple questions: Quick answers you can provide immediately
• Policy questions: Course-specific policies that AI won’t know
• Personal situations: Student accommodations or sensitive issues requiring human judgment

​

Reviewing AI Responses

• Check the evidence: Verify code snippets and line numbers are accurate
• Adjust the tone: Match how long you’ve been helping the student
• Verify against the spec: Ensure the advice aligns with assignment requirements
• Don’t give away solutions: Make sure hints guide without revealing answers

​

Effective Prompting

• Add context: Mention how long you’ve been helping the student
• Specify tone: Request Socratic questioning or direct hints
• Request specific analysis: Ask for focus on particular aspects
• Iterate: Follow up with additional questions as needed

​

Troubleshooting

​

”pawtograder” server not showing in Claude

1. Verify your claude_desktop_config.json syntax is valid
2. Check that your token hasn’t expired (tokens are valid for 90 days)
3. Ensure you replaced YOUR_TOKEN_HERE with your actual token
4. Restart Claude Desktop completely after configuration changes
5. Check Claude Desktop logs for connection errors

​

”Access denied” or “Invalid or expired API token” errors

• Verify you have instructor or grader role in the class
• Check that your API token hasn’t been revoked or expired
• Ensure your API token has the required scopes (mcp:read)
• Create a new token in Settings → API Tokens if needed

​

AI can’t access class data

• Verify you have instructor or grader permissions in the class
• The MCP server only works for instructors and graders, not students
• Check that you’re requesting data from the correct class ID

​

Token creation fails

• Verify you have instructor or grader role in at least one class
• Check that your token name is not empty and under 100 characters
• Ensure you’re not exceeding any token limits