Merge pull request #112 from sandialabs/copilot/report-rate-throttling-errors

Surface LLM errors to users with classified, actionable messages

Author: garland3

.env.example

@@ -8,7 +8,7 @@ MOCK_RAG=true
 
 # Server configuration
 PORT=8000
-APP_NAME=Chat UI 13
+APP_NAME=ATLAS
 
 # Authentication configuration
 # Header name to extract authenticated username from reverse proxy

IMPLEMENTATION_SUMMARY.md

@@ -0,0 +1,144 @@
+# Implementation Complete: Rate Limiting & Backend Error Reporting
+
+## ✅ Task Completed Successfully
+
+All backend errors (including rate limiting) are now properly reported to users with helpful, actionable messages.
+
+---
+
+## What Was Changed
+
+### 1. Error Classification System
+Created a comprehensive error detection and classification system that:
+- Detects rate limit errors (Cerebras, OpenAI, etc.)
+- Detects timeout errors
+- Detects authentication failures
+- Handles generic LLM errors
+
+### 2. User-Friendly Error Messages
+Users now see helpful messages instead of silence:
+
+| Situation | User Sees |
+|-----------|-----------|
+| Rate limit hit | "The AI service is experiencing high traffic. Please try again in a moment." |
+| Request timeout | "The AI service request timed out. Please try again." |
+| Auth failure | "There was an authentication issue with the AI service. Please contact your administrator." |
+| Other errors | "The AI service encountered an error. Please try again or contact support if the issue persists." |
+
+### 3. Security & Privacy
+- ✅ No sensitive information (API keys, internal errors) exposed to users
+- ✅ Full error details still logged for debugging
+- ✅ CodeQL security scan: 0 vulnerabilities
+
+---
+
+## Files Modified (8 files, 501 lines)
+
+### Backend Core
+- `backend/domain/errors.py` - New error types
+- `backend/application/chat/utilities/error_utils.py` - Error classification logic
+- `backend/main.py` - Enhanced WebSocket error handling
+
+### Tests (All Passing ✅)
+- `backend/tests/test_error_classification.py` - 9 unit tests
+- `backend/tests/test_error_flow_integration.py` - 4 integration tests
+
+### Documentation
+- `docs/error_handling_improvements.md` - Complete guide
+- `docs/error_flow_diagram.md` - Visual flow diagram
+- `scripts/demo_error_handling.py` - Interactive demonstration
+
+---
+
+## How to Test
+
+### 1. Run Automated Tests
+```bash
+cd backend
+export PYTHONPATH=/path/to/atlas-ui-3/backend
+python -m pytest tests/test_error_classification.py tests/test_error_flow_integration.py -v
+```
+**Result**: 13/13 tests passing ✅
+
+### 2. View Demonstration
+```bash
+python scripts/demo_error_handling.py
+```
+Shows examples of all error types and their user-friendly messages.
+
+### 3. Manual Testing (Optional)
+To see the error handling in action:
+1. Start the backend server
+2. Configure an invalid API key or trigger a rate limit
+3. Send a message through the UI
+4. Observe the error message displayed to the user
+
+---
+
+## Before & After Example
+
+### Before (The Problem)
+```
+User: *Sends a message*
+Backend: *Hits Cerebras rate limit*
+UI: *Sits there thinking... forever*
+Backend Logs: "litellm.RateLimitError: We're experiencing high traffic..."
+User: 🤷 "Is it broken? Should I refresh? Wait?"
+```
+
+### After (The Solution)
+```
+User: *Sends a message*
+Backend: *Hits Cerebras rate limit*
+UI: *Shows error message in chat*
+  "The AI service is experiencing high traffic. 
+   Please try again in a moment."
+Backend Logs: "Rate limit error: litellm.RateLimitError: ..."
+User: ✅ "OK, I'll wait a bit and try again"
+```
+
+---
+
+## Key Benefits
+
+1. **Better User Experience**: Users know what happened and what to do
+2. **Reduced Support Burden**: Fewer "why isn't it working?" questions
+3. **Maintained Security**: No sensitive data exposed
+4. **Better Debugging**: Full error details still logged
+5. **Extensible**: Easy to add new error types in the future
+
+---
+
+## What Happens Now
+
+The error classification system is now active and will:
+- Automatically detect and classify backend errors
+- Send user-friendly messages to the frontend
+- Log detailed error information for debugging
+- Work for any LLM provider (Cerebras, OpenAI, Anthropic, etc.)
+
+No further action needed - the system is ready to use!
+
+---
+
+## Documentation
+
+For more details, see:
+- `docs/error_handling_improvements.md` - Complete technical documentation
+- `docs/error_flow_diagram.md` - Visual diagram of error flow
+- Code comments in modified files
+
+---
+
+## Security Verification
+
+✅ CodeQL Security Scan: **0 alerts**  
+✅ Code Review: **All comments addressed**  
+✅ Tests: **13/13 passing**  
+✅ No sensitive data exposure verified
+
+---
+
+## Questions?
+
+See the documentation files or review the code comments for technical details. The implementation is thoroughly documented and tested.

agent_start.sh

@@ -24,9 +24,8 @@ cleanup_mcp() {
 }
 
 cleanup_processes() {
-    echo "Killing any running uvicorn processes for main backend... and python processes"
-    pkill -f "uvicorn main:app"
-    pkill -f python
+    echo "Killing any running uvicorn processes for main backend..."
+    pkill -f "uvicorn main:app" || true
     sleep 2
     clear
 }

backend/application/chat/service.py

@@ -13,6 +13,7 @@
     ToolResult
 )
 from domain.sessions.models import Session
+from domain.errors import DomainError
 from interfaces.llm import LLMProtocol, LLMResponse
 from interfaces.events import EventPublisher
 from interfaces.sessions import SessionRepository
@@ -262,7 +263,12 @@ async def handle_chat_message(
                 update_callback=update_callback,
                 **kwargs
             )
+        except DomainError:
+            # Let domain-level errors (e.g., LLM / rate limit / validation) bubble up
+            # so transport layers (WebSocket/HTTP) can handle them consistently.
+            raise
         except Exception as e:
+            # Fallback for unexpected errors in HTTP-style callers
             return error_utils.handle_chat_message_error(e, "chat message handling")
 
     async def handle_reset_session(

backend/application/chat/utilities/error_utils.py

@@ -6,9 +6,9 @@
 """
 
 import logging
-from typing import Any, Dict, List, Optional, Callable, Awaitable
+from typing import Any, Dict, List, Optional, Callable, Awaitable, Tuple
 
-from domain.errors import ValidationError
+from domain.errors import ValidationError, RateLimitError, LLMTimeoutError, LLMAuthenticationError, LLMServiceError
 from domain.messages.models import MessageType
 
 logger = logging.getLogger(__name__)
@@ -60,6 +60,42 @@ async def safe_get_tools_schema(
         raise ValidationError(f"Failed to get tools schema: {str(e)}")
 
 
+def classify_llm_error(error: Exception) -> Tuple[type, str, str]:
+    """
+    Classify LLM errors and return appropriate error type, user message, and log message.
+    
+    Returns:
+        Tuple of (error_class, user_message, log_message).
+
+    NOTE: user_message MUST NOT contain raw exception details or sensitive data.
+    """
+    error_str = str(error)
+    error_type_name = type(error).__name__
+    
+    # Check for rate limiting errors
+    if "RateLimitError" in error_type_name or "rate limit" in error_str.lower() or "high traffic" in error_str.lower():
+        user_msg = "The AI service is experiencing high traffic. Please try again in a moment."
+        log_msg = f"Rate limit error: {error_str}"
+        return (RateLimitError, user_msg, log_msg)
+    
+    # Check for timeout errors
+    if "timeout" in error_str.lower() or "timed out" in error_str.lower():
+        user_msg = "The AI service request timed out. Please try again."
+        log_msg = f"Timeout error: {error_str}"
+        return (LLMTimeoutError, user_msg, log_msg)
+    
+    # Check for authentication/authorization errors
+    if any(keyword in error_str.lower() for keyword in ["unauthorized", "authentication", "invalid api key", "invalid_api_key", "api key"]):
+        user_msg = "There was an authentication issue with the AI service. Please contact your administrator."
+        log_msg = f"Authentication error: {error_str}"
+        return (LLMAuthenticationError, user_msg, log_msg)
+    
+    # Generic LLM service error (non-validation)
+    user_msg = "The AI service encountered an error. Please try again or contact support if the issue persists."
+    log_msg = f"LLM error: {error_str}"
+    return (LLMServiceError, user_msg, log_msg)
+
+
 async def safe_call_llm_with_tools(
     llm_caller,
     model: str,
@@ -73,7 +109,7 @@ async def safe_call_llm_with_tools(
     """
     Safely call LLM with tools and error handling.
     
-    Pure function that handles LLM calling errors.
+    Pure function that handles LLM calling errors with proper classification.
     """
     try:
         if data_sources and user_email:
@@ -85,11 +121,13 @@ async def safe_call_llm_with_tools(
             llm_response = await llm_caller.call_with_tools(
                 model, messages, tools_schema, tool_choice, temperature=temperature
             )
-            logger.info(f"LLM response received with tools only, llm_response: {llm_response}")
+            logger.info("LLM response received with tools only, llm_response: %s", llm_response)
         return llm_response
     except Exception as e:
-        logger.error(f"Error calling LLM with tools: {e}", exc_info=True)
-        raise ValidationError(f"Failed to call LLM with tools: {str(e)}")
+        # Classify the error and raise appropriate error type
+        error_class, user_msg, log_msg = classify_llm_error(e)
+        logger.error(log_msg, exc_info=True)
+        raise error_class(user_msg)
 
 
 async def safe_execute_single_tool(

backend/domain/errors.py

@@ -46,6 +46,11 @@ class LLMError(DomainError):
     pass
 
 
+class LLMServiceError(LLMError):
+    """Generic LLM service failure that is not a validation issue."""
+    pass
+
+
 class ToolError(DomainError):
     """Tool execution error."""
     pass
@@ -74,3 +79,18 @@ class SessionNotFoundError(SessionError):
 class PromptOverrideError(DomainError):
     """Raised when MCP prompt override fails."""
     pass
+
+
+class RateLimitError(LLMError):
+    """Raised when LLM rate limit is exceeded."""
+    pass
+
+
+class LLMTimeoutError(LLMError):
+    """Raised when LLM request times out."""
+    pass
+
+
+class LLMAuthenticationError(AuthenticationError):
+    """Raised when LLM authentication fails."""
+    pass

backend/main.py

@@ -15,7 +15,13 @@
 from dotenv import load_dotenv
 
 # Import domain errors
-from domain.errors import ValidationError
+from domain.errors import (
+    ValidationError, 
+    RateLimitError, 
+    LLMTimeoutError, 
+    LLMAuthenticationError,
+    DomainError
+)
 
 # Import from core (only essential middleware and config)
 from core.middleware import AuthMiddleware
@@ -308,16 +314,47 @@ async def handle_chat():
                             update_callback=lambda message: websocket_update_callback(websocket, message),
                             files=data.get("files")
                         )
+                    except RateLimitError as e:
+                        logger.warning(f"Rate limit error in chat handler: {e}")
+                        await websocket.send_json({
+                            "type": "error",
+                            "message": str(e.message if hasattr(e, 'message') else e),
+                            "error_type": "rate_limit"
+                        })
+                    except LLMTimeoutError as e:
+                        logger.warning(f"Timeout error in chat handler: {e}")
+                        await websocket.send_json({
+                            "type": "error",
+                            "message": str(e.message if hasattr(e, 'message') else e),
+                            "error_type": "timeout"
+                        })
+                    except LLMAuthenticationError as e:
+                        logger.error(f"Authentication error in chat handler: {e}")
+                        await websocket.send_json({
+                            "type": "error",
+                            "message": str(e.message if hasattr(e, 'message') else e),
+                            "error_type": "authentication"
+                        })
                     except ValidationError as e:
+                        logger.warning(f"Validation error in chat handler: {e}")
+                        await websocket.send_json({
+                            "type": "error",
+                            "message": str(e.message if hasattr(e, 'message') else e),
+                            "error_type": "validation"
+                        })
+                    except DomainError as e:
+                        logger.error(f"Domain error in chat handler: {e}", exc_info=True)
                         await websocket.send_json({
                             "type": "error",
-                            "message": str(e)
+                            "message": str(e.message if hasattr(e, 'message') else e),
+                            "error_type": "domain"
                         })
                     except Exception as e:
-                        logger.error(f"Error in chat handler: {e}", exc_info=True)
+                        logger.error(f"Unexpected error in chat handler: {e}", exc_info=True)
                         await websocket.send_json({
                             "type": "error",
-                            "message": "An unexpected error occurred"
+                            "message": "An unexpected error occurred. Please try again or contact support if the issue persists.",
+                            "error_type": "unexpected"
                         })
 
                 # Start chat handling in background