Manually tracking expenses is tedious, and traditional budgeting apps require constant input, making them inconvenient. We wanted to create a seamless solution that integrates effortlessly into daily life. Inspired by the ubiquity of WhatsApp, we're thrilled to present Finance Copilot—a chatbot that lets users track expenses via text, images, or voice notes without opening another app. This is our official submission for Hack_NCState 2025!
Throughout this project, we gained hands-on experience with:
- Large Language Models (LLMs), specifically Google Gemini, for extracting financial data from user inputs.
- WhatsApp chatbot integration using Twilio API to facilitate smooth and responsive interactions.
- Django for server-side management, ensuring efficient request handling and transaction processing.
- MongoDB Atlas as a NoSQL database for scalable and efficient storage of financial data.
-
Tech Stack
- Backend: Django (Python) for handling chatbot interactions and transaction processing.
- AI Processing: Google Gemini API for understanding and categorizing transactions from text, images, and voice inputs.
- Database: MongoDB Atlas for storing user transactions and budgeting data.
- Messaging Platform: Twilio API for WhatsApp chatbot integration.
-
Key Features
- Users can log expenses via text, voice, or image.
- We categorize transactions with the help of Google Gemini and provide spending insights.
- A reward system incentivizes users to stay within budget.
- Processing multimodal inputs: Handling text, images, and voice required optimizing prompt engineering for Google Gemini.
- Ensuring accurate categorization: Fine-tuning LLM prompts to improve transaction classification.
- Seamless WhatsApp integration: Designing a conversational flow that extracts structured data while maintaining a natural user experience.
- Optimizing database queries: Structuring MongoDB documents efficiently for fast retrieval and analytics.
- Building a meaningful reward system: Creating an incentive mechanism that genuinely encourages better spending habits.
Finance Copilot transforms expense tracking by making it intuitive, automated, and rewarding. By leveraging LLMs, WhatsApp, and a robust backend, we eliminate manual entry and provide valuable financial insights.