New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(weave): refactor to use litellm instead of clients #3583

Merged

tcapelle merged 29 commits into master from scorers-litellm

Feb 6, 2025

Contributor

tcapelle commented Feb 4, 2025

Same as #3576 but only the litellm part
(suggested to split by @andrewtruong )

tcapelle requested a review from a team as a code owner

February 4, 2025 10:46

circle-job-mirror bot commented Feb 4, 2025 •

edited

Loading

Preview this PR with FeatureBee: https://beta.wandb.ai/?betaVersion=4e5508ddd150d7ca52793c66c830f4b12e12a3e3

Contributor Author

tcapelle commented Feb 4, 2025

Ok this is not going to work like this... Ideally Scorer act as an independent module that has no collision with weave/flow

tcapelle force-pushed the scorers-litellm branch from 24d1e29 to 5abf501 Compare

February 5, 2025 08:47

andrewtruong reviewed

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

andrewtruong reviewed

View reviewed changes

tests/scorers/test_llm_integrations.py Outdated Show resolved Hide resolved

andrewtruong reviewed

View reviewed changes

tests/scorers/test_ragas_scorer.py Show resolved Hide resolved

andrewtruong reviewed

View reviewed changes

weave/scorers/__init__.py Outdated Show resolved Hide resolved

andrewtruong reviewed

View reviewed changes

weave/scorers/llm_scorer.py Show resolved Hide resolved

andrewtruong reviewed

View reviewed changes

weave/scorers/llm_utils.py Outdated Show resolved Hide resolved

tcapelle force-pushed the scorers-litellm branch from 7830378 to d06c636 Compare

February 5, 2025 20:19

andrewtruong reviewed

View reviewed changes

weave/scorers/initialization.py Outdated Show resolved Hide resolved

andrewtruong approved these changes

View reviewed changes

tcapelle added 17 commits

February 6, 2025 14:57


          refactor to use litellm instead of clients

ffea9fc

correct import

rename models for litellm

make litellm default

one call per provider


          simplify

40911e0


          update deps

478e5e8


          update to use litellm acompletion

b1d6071


          remove as unnecesary

fbc6c9d


          better typing

3137fcb


          make method private

2bc2f2f


          lint

b230a02


          wrong import path

eda4dd6


          Trigger CI

8f53705


          gemini key

759ab17


          rename shard

523fffc


          sue a fixture

1c347b2


          move check

5358b2d


          rename the file containing the default models

87d06c6


          lint and clean up

ec95094


          deprecation warn

tcapelle added 8 commits

February 6, 2025 14:57


          change to flash

3af3ec2


          lint

45780ac


          move in the fixture

e4cec78


          mypy

3e027ab


          merge fixture

493b8a8

nit

67592be


          unify docstrings

235d442


          remove transpose, clean up

bdd4932

tcapelle force-pushed the scorers-litellm branch from f07138d to bdd4932 Compare

February 6, 2025 13:57

tcapelle added 4 commits

February 6, 2025 14:59


          remove transpose from here

3825bdb


          typo

8e3f642


          fix tests

5e4cc48


          renamed method

221c1bf

tcapelle merged commit f87794f into master

126 checks passed

tcapelle deleted the scorers-litellm branch

February 6, 2025 14:45

github-actions bot locked and limited conversation to collaborators

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet