[API View] Feedback mechanism for AI Generated comments #12786

AlitzelMendez · 2025-11-05T23:30:34Z

closes: #12464
Implements optional feedback collection when users downvote or delete AI-generated comments to help improve AI comment quality.

When users interact negatively with AI-generated comments (downvote 👎 or delete 🗑️), a feedback dialog appears asking why. Users must select at least one reason from 4 predefined options and can optionally provide additional details.

Feedback Options:

Information is factually incorrect
APIView tool limitation or quirk
Azure SDK design decision/guideline
Other

Key Behavior

✅ Action executes immediately (downvote/delete happens first)
✅ Feedback dialog appears afterward
✅ Users can skip by clicking Cancel
✅ Feedback stored with timestamp and user attribution

Changes

Frontend:

New reusable feedback dialog component
Optimized comment thread state management
Full dark mode support

Backend:

New API endpoint: POST /comments/{reviewId}/{commentId}/feedback
Feedback stored on CommentItemModel.Feedback property
Includes comprehensive unit tests

Demo

feedback_demo.mp4

Copilot

Pull Request Overview

This PR implements a feedback collection mechanism for AI-generated comments in the API review system. When users downvote or delete AI-generated comments, they are prompted to provide feedback about why the comment was unhelpful.

Key changes:

Added a new feedback dialog component for collecting user feedback on AI comments
Integrated feedback collection into the downvote and delete workflows for AI-generated comments
Created backend API endpoint and data models to store feedback

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
ai-comment-feedback-dialog.component.ts	New component implementing the feedback dialog UI and logic
ai-comment-feedback-dialog.component.html	Template for the feedback dialog with checkboxes and text input
ai-comment-feedback-dialog.component.scss	Styling for the feedback dialog
ai-comment-feedback-dialog.component.spec.ts	Unit tests for the feedback dialog component
comment-thread.component.ts	Modified to intercept AI comment downvotes/deletes and show feedback dialog
comment-thread.component.html	Added feedback dialog to the comment thread template
comments.service.ts	Added service method to submit feedback to backend API
review-page-layout.module.ts	Registered the new feedback dialog component
commentThreadUpdateDto.ts	Added AICommentFeedback interface
CommentsController.cs	Added API endpoint for submitting comment feedback
CommentsManager.cs	Added manager method to process and store feedback
CommentItemModel.cs	Added CommentFeedback model to store feedback data
CommentFeedbackRequest.cs	New request model for feedback API
ICommentsManager.cs	Added interface method for feedback submission

...tnet/APIView/ClientSPA/src/app/_components/shared/comment-thread/comment-thread.component.ts

…t-thread/comment-thread.component.ts Co-authored-by: Copilot <[email protected]>

…lMendez/azure-sdk-tools into downvote--delete-feedback

AlitzelMendez · 2025-11-08T01:42:28Z

/azp run

azure-pipelines · 2025-11-08T01:42:35Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

AlitzelMendez · 2025-11-08T01:43:12Z

/azp run APIView-CI

azure-pipelines · 2025-11-08T01:43:18Z

No pipelines are associated with this pull request.

AlitzelMendez · 2025-11-08T01:43:51Z

/azp run APIView - CI

azure-pipelines · 2025-11-08T01:44:02Z

Azure Pipelines successfully started running 1 pipeline(s).

tjprescott · 2025-11-11T20:55:29Z

@AlitzelMendez this is no longer true, right?

tjprescott

I think we should store the "reasons" as enums. The dialog should also appear when using bulk resolution (currently on UX Test it doesn't).

tjprescott · 2025-11-11T20:58:04Z

src/dotnet/APIView/APIViewWeb/LeanModels/CommentItemModel.cs

+        public string Comment { get; set; } = string.Empty;
+        public bool IsDelete { get; set; }
+        public string SubmittedBy { get; set; }
+        public DateTime? SubmittedOn { get; set; }


Why is this optional?

tjprescott · 2025-11-11T20:58:59Z

src/dotnet/APIView/APIViewWeb/LeanModels/CommentItemModel.cs

        public List<string> MemoryIds { get; set; } = [];
        public float ConfidenceScore { get; set; }

+        public List<CommentFeedback> Feedback { get; set; } = [];


This should probably be nullable. Since it only even applies to AI-generated comments, there's no point storing even empty list for non-AI comments.

tjprescott · 2025-11-11T20:59:53Z

src/dotnet/APIView/APIViewWeb/LeanModels/CommentFeedbackRequest.cs

+{
+    public class CommentFeedbackRequest
+    {
+        public List<string> Reasons { get; set; } = new();


My gut tells me we should store these as enums, as that will allow copilot to take deterministic action instead of having to parser intent.

tjprescott · 2025-11-11T21:02:27Z

...rc/app/_components/shared/ai-comment-feedback-dialog/ai-comment-feedback-dialog.component.ts

+  readonly feedbackReasons = [
+    'This comment is factually incorrect',
+    'This is an APIView rendering bug',
+    'This is an accepted APIView rendering choice',
+    'This is an accepted SDK design pattern',
+    'The guideline cited here is out-of-date'
+  ];


I think we should map these to enum values and store them as such.

AlitzelMendez · 2025-11-11T23:32:03Z

I think we should store the "reasons" as enums. The dialog should also appear when using bulk resolution (currently on UX Test it doesn't).

like when clicking thumbs down inside the bulk resolution dialog, trigger this feedback pop up?

@ccbarragan could you provide your feedback in here? what would be a good way to integrate this on the bulk resolution?
I have the feeling that triggering another pop up is to many layers of interaction, but is this, ok? or do you have another suggestion on how to add this?

tjprescott · 2025-11-11T23:46:53Z

like when clicking thumbs down inside the bulk resolution dialog, trigger this feedback pop up?

Yes. Bulk resolution is supposed to be the same as repeating the process for each item, so if downvoting an individual item triggers the dialogue, then so should downvoting a batch.

AlitzelMendez · 2025-11-12T00:33:49Z

Yes. Bulk resolution is supposed to be the same as repeating the process for each item, so if downvoting an individual item triggers the dialogue, then so should downvoting a batch.

Oh yes! I just want to think about ways to implement this… Should we create another pop-up? Expand a window inside the same pop-up? Add a small window to the right? Remove the option to provide more comments (since they already have the comment window) and instead show a small, clean window with options stating that it’s required to select at least one option—but they don’t need to click Submit? well but they can select more than one!

I will try to mock this ideas tomorrow morning

AlitzelMendez · 2025-11-12T01:21:45Z

@ccbarragan here are some suggestions/questions

Inline Feedback Section

In this one we have the control of, if you selected downvote you have to provide a reason, the button gets disable after clicking downvote.

It makes the experience harder, but we guarantee feedback if thumb down is provided

bulk_resolve_feedback_option1.mp4

Feedback After Resolve

This one is a more common pattern, but I think everyone is going to close it, not providing feedback and we will be on a weird place of, should we not downvote then? which is not going to be clear/easily visible

bulk_resolve_feedback_option2.mp4

Layered Dialog

Now that I see implemented it looks good! :) but still I have the question if this is a good/acceptable user experience.

The workflow would be similar at the current one, if not reason provided.... the thumbs downs don't persist

bulk_resolve_feedback_option3.mp4

tjprescott · 2025-11-12T16:47:15Z

@AlitzelMendez thanks for these! I think we should consider whether we want bulk operations to be like one big transaction or discrete. The answer I think helps inform which approach is the "right" one.

ccbarragan · 2025-11-12T22:13:19Z

@AlitzelMendez awesome explorations! shared some thoughts at our UX sync today. My preference on the three would be the first option inline feedback section, but displaying the feedback checkbox list after message / thumbs down / input field (instead of above).

AlitzelMendez · 2025-11-21T23:34:50Z

Updating with offline conversations to have the whole context here

For delete operation we want to always request a reason, if a reason is not provided, we should not delete

We should have a similar experience on bulk resolution

Add comment feedback collection

99b2ac3

Copilot AI review requested due to automatic review settings November 5, 2025 23:30

AlitzelMendez requested review from helen229, maririos, praveenkuttappan and tjprescott as code owners November 5, 2025 23:30

Copilot AI reviewed Nov 5, 2025

View reviewed changes

...tnet/APIView/ClientSPA/src/app/_components/shared/comment-thread/comment-thread.component.ts Outdated Show resolved Hide resolved

AlitzelMendez and others added 4 commits November 5, 2025 15:40

Update src/dotnet/APIView/ClientSPA/src/app/_components/shared/commen…

db837e6

…t-thread/comment-thread.component.ts Co-authored-by: Copilot <[email protected]>

Redesign

f642729

Merge branch 'downvote--delete-feedback' of https://github.com/Alitze…

d6244b5

…lMendez/azure-sdk-tools into downvote--delete-feedback

Merge branch 'main' into downvote--delete-feedback

fbbddce

AlitzelMendez added 2 commits November 10, 2025 10:06

Merge branch 'main' into downvote--delete-feedback

a9173dd

Feedbacl

2e93216

tjprescott requested changes Nov 11, 2025

View reviewed changes

[API View] Feedback mechanism for AI Generated comments #12786

Are you sure you want to change the base?

[API View] Feedback mechanism for AI Generated comments #12786

Uh oh!

Conversation

AlitzelMendez commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Key Behavior

Changes

Demo

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

AlitzelMendez commented Nov 8, 2025

Uh oh!

azure-pipelines bot commented Nov 8, 2025

Uh oh!

AlitzelMendez commented Nov 8, 2025

Uh oh!

azure-pipelines bot commented Nov 8, 2025

Uh oh!

AlitzelMendez commented Nov 8, 2025

Uh oh!

azure-pipelines bot commented Nov 8, 2025

Uh oh!

tjprescott commented Nov 11, 2025

Uh oh!

tjprescott left a comment

Choose a reason for hiding this comment

Uh oh!

tjprescott Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

tjprescott Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

tjprescott Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

tjprescott Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

AlitzelMendez commented Nov 11, 2025

Uh oh!

tjprescott commented Nov 11, 2025

Uh oh!

AlitzelMendez commented Nov 12, 2025

Uh oh!

AlitzelMendez commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Inline Feedback Section

Feedback After Resolve

Layered Dialog

Uh oh!

tjprescott commented Nov 12, 2025

Uh oh!

ccbarragan commented Nov 12, 2025

Uh oh!

AlitzelMendez commented Nov 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

AlitzelMendez commented Nov 5, 2025 •

edited

Loading

AlitzelMendez commented Nov 12, 2025 •

edited

Loading