feat(providers): formalize service providers as pluggable provider backends

[KooshaPari]

Summary

9router and CLIProxyAPI are currently wired as special-cased runtime services in several places (manifest injection, provider models routing, and UI/service endpoints). This has started to become a scalability bottleneck for router architecture work.

Requested change

• Introduce a shared provider-plugin contract for embedded services (service id, expose toggle, model sync, endpoint metadata).
• Register 9router/cliproxyapi providers through that path instead of duplicated service-specific behavior.
• Keep current providerExpose behavior (boolean) but make backend/provider registration data-driven.
• Preserve backward-compatibility with open-sse manifest + model prefixing semantics while decoupling from fixed if rawProvider in {...} checks.

Why now

• 9router and cliproxyapi need parity for future replacement with Go/Rust sidecars or external SDK-based routing.

[diegosouzapw]

Thanks for framing this as the broader architecture play — it aligns with the project's 3.9→4.0 modular roadmap and we've cataloged it internally.

One sequencing note: this overlaps with your own #7325 / PR #7324 (the narrower immediate slice). We'd like to land #7325 first, then treat this as the follow-up contract work generalizing that slice — let us know if you'd rather sequence it the other way.

Status: ✅ Accepted — queued for implementation (sequenced after #7325)

[diegosouzapw]

🚧 PR opened for release/v3.8.49!

Your feature request has an implementation ready — PR #7730 is open and awaiting the maintainer's review.

What was done:

• A pluggable ServiceProviderPlugin contract, with 9router migrated onto it as the proof. Fully backward compatible (regression-tested); cliproxyapi/mux/bifrost migration and the manifest wiring are deferred follow-ups
• Ships with new behavior tests (no existing test assertion was removed or weakened)

This will ship once the PR is merged and included in the upcoming v3.8.49 release. 🚀

[diegosouzapw]

✅ Partially shipped — PR #7730 is merged into release/v3.8.49.

What shipped:

• The pluggable ServiceProviderPlugin contract, with 9router migrated onto it as the proof — fully backward compatible

Kept open because this remains:

• Migrating cliproxyapi, generalizing mux/bifrost, consolidating the executor-routing special-casing, and wiring the manifest helper into a live /v1/models path

That's why the PR used Refs rather than Closes — the remaining scope is real work, not an oversight. Leaving this open to track it. 🙏