Skip to content

feat: auto-fill custom provider prices from models.dev#2024

Merged
guillaumegay13 merged 4 commits into
mnfst:mainfrom
guillaumegay13:feat/modelsdev-custom-provider-prices
May 26, 2026
Merged

feat: auto-fill custom provider prices from models.dev#2024
guillaumegay13 merged 4 commits into
mnfst:mainfrom
guillaumegay13:feat/modelsdev-custom-provider-prices

Conversation

@guillaumegay13
Copy link
Copy Markdown
Collaborator

@guillaumegay13 guillaumegay13 commented May 26, 2026

Summary

  • Index all chat-capable models.dev providers for custom-provider provider/model lookups.
  • Auto-fill missing custom-provider input/output prices and context windows during probe/create/update when provider and model match models.dev.
  • Fall back to exact model-ID matches from models.dev when the custom provider is not listed there, mark those imported prices as estimated, and show one estimate note beside the Models section.
  • Pass provider_name from the frontend probe flow so detected models return imported prices in the custom-provider form.

Context

This is the second step after the model catalog cap increase. It preserves manual prices/context, leaves unmatched model IDs on the existing blank-price/default-context path, and only uses exact models.dev matches. It does not fuzzy-match or invent prices.

As of 2026-05-26, models.dev has no Mammouth/Mammoth provider. A Mammouth-style custom provider can still prefill prices for exact model IDs that models.dev knows, such as openai/gpt-4o-mini; unknown Mammouth-native model IDs remain blank.

Validation

  • npm run build --workspace=packages/shared
  • npm test --workspace=packages/backend -- models-dev-sync.service.spec.ts custom-provider.service.spec.ts custom-provider.controller.spec.ts
  • npm test --workspace=packages/frontend -- CustomProviderForm.test.tsx api-extra.test.ts routing-extra.test.ts
  • npm test --workspace=packages/frontend -- CustomProviderForm.test.tsx InfoTooltip.test.tsx api-extra.test.ts routing-extra.test.ts
  • npm run build --workspace=packages/backend
  • npm run build --workspace=packages/frontend (existing large Playground chunk warning)
  • npm exec eslint -- packages/frontend/src/components/CustomProviderForm.tsx packages/frontend/src/components/InfoTooltip.tsx packages/frontend/src/services/api/routing.ts
  • npm exec prettier -- --check packages/frontend/src/components/CustomProviderForm.tsx packages/frontend/src/components/InfoTooltip.tsx packages/frontend/src/styles/routing-providers.css
  • git diff --check

Summary by cubic

Auto-fill custom provider model prices and context from models.dev using provider-name + model ID matches, with an exact model-only fallback marked as estimated. The form shows a single estimated-price tooltip beside Models, preserves user edits, and passes price_estimated through the API.

  • New Features
    • Indexes all chat-capable models.dev providers (including non-native) for custom-provider lookups by display name/alias.
    • Fills missing input_price_per_million_tokens, output_price_per_million_tokens, and context_window on create/update/probe when matched; preserves user-entered values; leaves unmatched blank.
    • Adds exact model-only price fallback across providers (prefers official catalogs first) and flags these with price_estimated: true.
    • Adds price_estimated to DTO/entity and API in packages/backend and packages/frontend; probe accepts provider_name and returns enriched models.
    • Frontend sends provider_name during probe, renders one info tooltip for estimated prices, and submits price_estimated when applicable.

Written for commit 174d329. Summary will update on new commits. Review in cubic

@codecov
Copy link
Copy Markdown

codecov Bot commented May 26, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.41%. Comparing base (3f58729) to head (174d329).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #2024   +/-   ##
=======================================
  Coverage   99.41%   99.41%           
=======================================
  Files         186      186           
  Lines       17321    17363   +42     
  Branches     6871     6884   +13     
=======================================
+ Hits        17220    17262   +42     
  Misses         99       99           
  Partials        2        2           
Flag Coverage Δ
frontend 99.46% <100.00%> (+<0.01%) ⬆️
shared 97.85% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@guillaumegay13 guillaumegay13 marked this pull request as ready for review May 26, 2026 07:35
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 13 files

Re-trigger cubic

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 11 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread .changeset/modelsdev-custom-provider-prices.md Outdated
@codecov
Copy link
Copy Markdown

codecov Bot commented May 26, 2026

Bundle Report

Changes will increase total bundle size by 2.04kB (0.1%) ⬆️. This is within the configured threshold ✅

Detailed changes
Bundle name Size Change
manifest-frontend-esm 2.06MB 2.04kB (0.1%) ⬆️

Affected Assets, Files, and Routes:

view changes for bundle: manifest-frontend-esm

Assets Changed:

Asset Name Size Change Total Size Change (%)
assets/index-*.css 449 bytes 221.66kB 0.2%
assets/ProviderSelectModal-*.js 897 bytes 117.83kB 0.77%
assets/index-*.js 200 bytes 57.48kB 0.35%
assets/routing-*.js 1.93kB 35.88kB 5.68% ⚠️
assets/routing-*.js 18 bytes 3.93kB 0.46%
assets/providers-*.js 81 bytes 6.14kB 1.34%
assets/model-*.js -1.53kB 488 bytes -75.83%

Files in assets/ProviderSelectModal-*.js:

  • ./src/components/CustomProviderForm.tsx → Total Size: 20.12kB

Files in assets/routing-*.js:

  • ./src/components/InfoTooltip.tsx → Total Size: 3.06kB

Files in assets/routing-*.js:

  • ./src/services/api/routing.ts → Total Size: 6.96kB

@guillaumegay13 guillaumegay13 merged commit ba7b3c6 into mnfst:main May 26, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant