Server-sent events fired 1000s times per conversation

[alex-mcgovern]

Describe the issue

Found it hard to export the logs from the server sent events, but here is a video recording of the network tab. This is for a single chat message that produced a handful of alerts.

Screen.Recording.2025-02-05.at.2.21.19.PM.mov

Steps to Reproduce

• Open a connection with /api/v1/alerts_notification
• chat with an LLM
• notice how many events are sent over the stream

Operating System

MacOS (Arm)

IDE and Version

Visual Studio Code version: 1.96.4

Extension and Version

Identifier github.copilot-chat Version 0.23.2

Provider

GitHub Copilot

Model

GPT 4o mini

Codegate version

CodeGate version: v0.1.16-5abdad6-dev

Logs

No response

Additional Context

No response

[lukehinds]

@aponcedeleonch might have some insight

[aponcedeleonch]

Hypothesis: What I think is going on is that we switched from doing simple inserts to doing upserts for Copilot because of how we receive messages. Before, we weren't storing everything (hence the switch), but now a single chat request can involve multiple upserts. Each time we do an upsert, we're sending an SSE. The alerts and everything else should stay the same, and they’re not actually being updated in the database—but we're still sending a bunch of SSEs.

This needs to be confirmed but if that's the case this is a Copilot specific problem. I think the table that needs upserting is outputs table. Right now, when upserting we upsert all 3 tables (prompts, alerts, outputs). If the hypothesis is confirmed we could change the logic to only upsert the outputs table.

[yrobla]

We changed this logic because with the caching, we were not recording the complete output of the message but just the initial fragment. Then the conversations in the UI didn't make sense because only the first chunk of the conversation was seen. I actually saw the problem in prompts table as well, that's why i modified.
But is true that now with multiple requests, there are multiple calls to DB. That has 2 problems :

• heavy load on the db
• multiple SSE calls

For the SSE calls we could implement some kind of debounce, but for the DB calls we should find another solution