Skip to content

OpenAI model cost based on model name fix #1946

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

JonasElburgUVA
Copy link
Contributor

the CostCallbackHandler.total_cost method can take a dictionary of models and associated cost per i/o tokens. However, this was not functional because the token usage parser did not save the model's name. The model's name was also removed after adding TokenUsage objects together. I fixed these issues for the OpenAI parser.

Note that the same problem still exists for the anthropic and bedrock parsers. I did not update this, since I don't use these API's and would not be able to test my implementation.

@dosubot dosubot bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label Mar 4, 2025
@jjmachan
Copy link
Member

jjmachan commented Mar 4, 2025

thanks for bringing this up @JonasElburgUVA I'll fix the others too after merging this but could you check the tests too?

seems like some tests were also incorrect

@JonasElburgUVA JonasElburgUVA marked this pull request as draft March 7, 2025 08:14
@JonasElburgUVA
Copy link
Contributor Author

Sorry for the failing tests in the initial PR. The test_cost.py tests did not expect a model ID as part of TokenUsage objects, even though the LLMResult samples do contain model IDs. I updated the tests accordingly. I also added the model ID parsing logic for the other parsers based on the LLMResult samples in test_cost.py.

codestyle check is still failing, I am not sure why?

@JonasElburgUVA JonasElburgUVA marked this pull request as ready for review March 10, 2025 08:37
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:XS This PR changes 0-9 lines, ignoring generated files. labels Mar 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:M This PR changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants