User token report

[paullizer]

Overview

Enhance the user profile section by including an estimated token usage report. This feature will provide users with visibility into their token consumption, helping them monitor AI model usage and optimize their interactions.

Feature Scope

1. Display Token Usage in User Profile

   • Show an estimated token count for each user in their profile section.
   • Provide breakdowns based on recent queries, total usage, and session-based consumption.

2. Real-Time or Periodic Updates

   • Implement a real-time counter or periodic updates to reflect token usage.
   • Ensure efficient calculation to prevent performance overhead.

3. Usage Insights & Limits (Optional)

   • Display warnings or recommendations if a user is nearing a token threshold.
   • Provide insights into high-usage queries to help users optimize their prompts.

Implementation Considerations

• Integrate with AI model response metadata to extract token usage per request.
• Store token estimates in Cosmos DB or another appropriate data store.
• Ensure accuracy and scalability by optimizing token counting logic.

This feature will improve transparency in AI interactions, helping users track and manage their token consumption effectively.