Skip to content

Conversation

@maxbrunsfeld
Copy link
Collaborator

  • capture and store teacher model's predicted cursor position
  • provide cursor position to student during distillation
  • eval cursor positions
  • parse and apply cursor position predictions at runtime

Release Notes:

  • N/A

- Add cursor_distance and cursor_exact_match fields to ExampleScore
- Compute cursor metrics by comparing expected vs actual cursor positions
- For Teacher prompts, apply expected patch to editable region to find
  hunk offset, then compute expected cursor position relative to the
  editable region (matching how actual cursor offset is stored)
- Use cleaned patches from expected_patches_with_cursor_positions() to
  avoid issues with CURSOR_POSITION comment lines in patches
- Add cursor metrics to print_report() showing exact match (✓),
  distance (±N), miss (✗), or skipped (-)
- Add cursor_exact_match_rate, cursor_avg_distance, and
  cursor_total_evaluated to SummaryJson
- Make TeacherPrompt::extract_editable_region public for use in scoring
- Fix unwrap() panic in TeacherPrompt::parse when editable region not found
@cla-bot cla-bot bot added the cla-signed The user has signed the Contributor License Agreement label Jan 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed The user has signed the Contributor License Agreement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants