feat: trigger Scout on issue comments and model threads as conversations#27
Open
dsblank wants to merge 2 commits into
Open
feat: trigger Scout on issue comments and model threads as conversations#27dsblank wants to merge 2 commits into
dsblank wants to merge 2 commits into
Conversation
Scout now runs on new issue comments, not only on issue open, and reads the whole thread as a multi-turn conversation. - Trigger: workflows listen on `issue_comment: [created]` in addition to `issues: [opened]`. main() gates comment runs — skips PR comments, Scout's own comments (hidden marker), and bots; optional SCOUT_COMMENT_TRIGGER_MENTION restricts to @scout mentions. Reacts on the triggering comment. - Conversation: build_conversation() renders the issue + thread as alternating user/assistant turns. Human turns are prefixed [author (association)] so the model can weigh maintainer input; Scout's prior comments become assistant turns. Consecutive same-role turns are merged for valid alternation. - Author association: get_issue_data() now returns author_association and per-comment association + role on both providers. - Opik threads: each run records just its turn (latest comment in -> reply out) under the existing per-issue thread_id, so a thread reads as a clean dialogue. - Provider parity: both providers now keep the most recent 20 comments (was first 20 — would drop the triggering comment) and truncate comment bodies to 500 chars, so evals exercise the production code path. - Simulator/scenarios/tests/docs updated: add_comment() builder, a multi-party comment-thread starter scenario, and parity tests asserted against both providers. System prompt and READMEs document the new behavior. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Scout runs on GITHUB_TOKEN and now reacts to comments and reads threads. When the token lacks the required permissions, the pre-agent GitHub calls (provider setup, reaction, get_issue_data) previously crashed with a raw traceback. - Wrap provider construction and get_issue_data so a 401/403 exits non-zero with a clear message naming the permissions the workflow must grant (issues: write, contents: read). - _react() makes the 👀 acknowledgement best-effort: a missing reaction permission is logged, not fatal, and a failed comment reaction falls back to the issue. Triage proceeds regardless. - The top-level handler now renders GithubException with the same actionable message instead of a traceback (e.g. a permission error posting the comment). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Scout previously ran only when an issue was opened. This adds a second trigger — new comments — and reworks how an issue is presented to the model so a comment thread is read as a real conversation.
Changes
Trigger & gating (
triage.py, workflows)issue_comment: [created]in addition toissues: [opened]._should_skip_comment_event()skips: comments on PRs, Scout's own comments (detected by the hidden<!-- scout-feedback -->marker → loop prevention), and bot accounts.SCOUT_COMMENT_TRIGGER_MENTION=truerestricts comment runs to those that@scout.Conversation modeling (
agent.py)build_issue_message→build_conversation(): issue + comments become alternatinguser/assistantturns. Human turns are prefixed[author (association)]:; Scout's prior comments map toassistantturns. Consecutive same-role turns are merged so the Messages API sees valid alternation.Author association (
providers/github.py,simulator.py)get_issue_data()now returnsauthor_associationand per-commentassociation+role. Marker detection runs on the full body before truncation.Opik threads (
agent.py)thread_id; each run now records just its own turn (latest comment in → reply out) so the thread reads as a clean dialogue instead of an ever-growing transcript. The model still sees the full conversation.Provider parity (the simulator must mirror production or evals are meaningless)
Tests / scenarios / docs
add_comment()simulator builder, a multi-party comment-thread starter scenario, and parity tests. System prompt + both READMEs updated. 144 tests pass; ruff + mypy clean.Workflow YAMLs touched / affected
.github/workflows/test.yml— addedissue_comment: [created]trigger, updatedconcurrencygroup andISSUE_NUMBERto resolve from either event (dogfoods the comment path on this repo).issue_commenttrigger (this is what consumers copy).action.yml/actions/feedback/action.yml— no change needed; triggers live in the caller workflow and the new env var propagates through step-levelenv:.Breaking changes
build_issue_message()is replaced bybuild_conversation()(returns(messages, latest_turn)).get_issue_data()dict gainsauthor_associationand per-commentassociation/role.GitHubProvider.add_comment_reaction(issue_number, comment_id, reaction)added.🤖 Generated with Claude Code