Skip to content

Conversation

@embire2
Copy link

@embire2 embire2 commented Sep 7, 2025

Description

This PR implements a minimal solution to prevent chat conversations from hanging when the AI stream becomes unresponsive.

Problem

Users experience hanging chat conversations when:

  • The AI stream becomes unresponsive
  • Network issues cause stream interruptions
  • API timeouts occur without proper handling

Solution

Implemented a lightweight StreamRecoveryManager that:

  • Monitors stream activity with 45-second timeout
  • Automatically attempts recovery (2 retries)
  • Provides proper cleanup on stream completion
  • Logs timeout events for debugging

Changes (2 files only)

  1. app/lib/.server/llm/stream-recovery.ts (new file, 92 lines)

    • StreamRecoveryManager class
    • Timeout detection and recovery logic
  2. app/routes/api.chat.ts (15 line additions)

    • Import StreamRecoveryManager
    • Initialize recovery manager
    • Start monitoring on stream execute
    • Update activity in stream loop
    • Stop monitoring on completion

Testing

✅ Tested with various AI models
✅ Verified timeout detection works
✅ Confirmed proper cleanup
✅ Normal conversations unaffected

Impact

  • Minimal: Only 2 files, 107 lines total
  • Non-breaking: Backward compatible
  • Performance: Negligible overhead

Fixes #1964


Author: Keoma Wright

- Add StreamRecoveryManager for handling stream timeouts
- Monitor stream activity with 45-second timeout
- Automatic recovery with 2 retry attempts
- Proper cleanup on stream completion

Fixes stackblitz-labs#1964
@Stijnus
Copy link
Collaborator

Stijnus commented Sep 7, 2025

Thank you , Good job!

@Stijnus Stijnus merged commit 2fde6f8 into stackblitz-labs:main Sep 7, 2025
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error in chat conversations. Don't move forward.

2 participants