feat: nvidia NIM local RPM budget + concurrency cap (#6846)

[diegosouzapw]

Summary

Phase 1 of client-side quota tracking for the nvidia (NVIDIA NIM) provider, which sends no rate-limit headers and has no usage API (see #6846's citations).

• Static local RPM budget: registered nvidia: { requests: 40, windowMs: 60_000 } in PROVIDER_DEFAULT_RATE_LIMITS (open-sse/services/providerDefaultRateLimit.ts), matching the documented "~40 RPM" free-tier note. Already wired end-to-end through the existing withRateLimit() → awaitProviderDefaultSlot() pre-schedule gate — no new call site needed.
• Operator override: added ResilienceSettings.providerQuotaOverrides: Record<string, { rpm?, concurrency? }> (types/normalize/settings.ts), resolved in rateLimitManager.ts::initializeRateLimitProtection(). Not yet exposed via the /api/resilience route or a dashboard form (no UI in Phase 1) — the mechanism is real and tested, wiring the UI is Phase 3 scope.
• Per-connection concurrency cap (default 6, mid-point of the issue's suggested 4-8 range): new leaf module open-sse/executors/default/nvidiaConcurrencyGate.ts wraps the existing rateLimitSemaphore.ts, wired into DefaultExecutor.execute() (the only dispatch path nvidia runs through — no dedicated nvidia executor exists). No-op for every other provider.
• Per-model 429 lockout: investigated during implementation and found already satisfied — issue fix(nvidia): NIM models listed available:true but 404 at router (catalog/routing disagree) #6773 (landed after this plan's research) already set passthroughModels: true on nvidia's registry entry, so hasPerModelQuota("nvidia") already returns true via the existing generic passthrough branch. No code change needed; added a regression-guard test instead of redundant code.
• Docs: docs/reference/PROVIDER_REFERENCE.md is auto-generated ("do not edit by hand"), so added a manual "Provider-specific quota tracking notes" section instead of hand-editing the generated table.

Deferred (explicitly out of scope for this PR, per the plan's own scope note)

• Phase 2 — AIMD adaptive per-model ceiling learning from 429 frequency, Retry-After honoring, persistence design.
• Phase 3 — dashboard "Provider Quota" card, combo-routing headroom-based candidate filtering/rotation, /api/resilience exposure of providerQuotaOverrides.

This PR is Phase 1 only — the issue stays open for the Phase 2/3 follow-ups.

How this was validated (Hard Rule #18 — TDD)

New test file tests/unit/nvidia-quota-phase1.test.ts (13 tests, mocked/synthetic fixtures only — no live NIM calls):

• RPM budget: 40/60s default, 41st request throttled; per-provider RPM override takes precedence; override doesn't leak to other providers.
• Per-model lockout: hasPerModelQuota("nvidia") true; a 429 lock on one model doesn't lock a sibling model on the same connection.
• Concurrency cap: 3rd concurrent request at cap=2 queues and resolves once a slot frees; cap is scoped per-connection (two different nvidia connections don't share a gate); no-op for non-nvidia providers / missing connectionId; override resolution order (override → static default 6 → fallback).
• Failure-mode separation regression guard: static assertion that PROVIDER_BREAKER_FAILURE_STATUSES in src/sse/handlers/chat.ts (untouched by this PR) still excludes 429.

Proved red→green for the actually-new code: temporarily reverted the nvidia map entry in providerDefaultRateLimit.ts, confirmed the "real default RPM budget" test failed, restored it and confirmed all 13 tests pass. (The hasPerModelQuota test intentionally does NOT prove red→green against my diff, since that requirement was already met by #6773 — see commit message / plan-file note.)

Gates run locally, all green: check-test-discovery, typecheck:core, typecheck:noimplicit:core (pre-existing unrelated errors in combo.ts/usageTracking.ts/cliRuntime.ts confirmed untouched by this diff), eslint on all changed files, check:complexity-ratchets (no regression vs baseline), check-file-size (one documented rebaseline: open-sse/executors/default.ts 877→890, justified in file-size-baseline.json), check:cycles, check:docs-sync, check:any-budget:t11, check:tracked-artifacts. Also re-ran the pre-existing rate-limit-provider-default.test.ts, account-fallback-service.test.ts, nvidia-passthrough-models-6773.test.ts, rate-limit-manager.test.ts, and the full resilience-settings-*.test.ts family — all green (one pre-existing test's exact-key-list assertion was updated to include the new providerQuotaOverrides section).

test:coverage was not re-run locally per the task's own instructions (heaviest gate, shared box) — this change is additive-only (new leaf module + new branches gated behind provider === "nvidia"), fully covered by the new test file; CI is authoritative.

Refs #6846

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!