Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OVHcloud as an inference provider #1303

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

fabienric
Copy link

What

Adds OVHcloud as an inference provider.

Test Plan

Added new tests for OVHcloud both with and without streaming.

What Should Reviewers Focus On?

I used the Cerebras PR as an example.

@julien-c julien-c added the inference-providers integration of a new or existing Inference Provider label Apr 2, 2025
@julien-c
Copy link
Member

julien-c commented Apr 2, 2025

Hi @fabienric we are currently finishing a refactoring of Inference Providers integration code in #1315, this should be merged soon, but we will need to rewrite part of your implementation (should be even simpler to integrate), will ping again after it's been merged.

@hanouticelina
Copy link
Contributor

Hi @fabienric,
We've merged a refactoring for Inference Provider integration into main, which should make adding new providers much easier.
Could you merge main into your branch and update your PR accordingly? it should be relatively straightforward with the new structure:
1 - You have to update the PROVIDERS mapping here: inference/src/lib/getProviderHelper.ts#L49 and add ovhcloud (let's ensure we respect the alphabetical order) :

import * as OvhCloud from "../providers/ovhcloud";
...
export const PROVIDERS: Record<InferenceProvider, Partial<Record<InferenceTask, TaskProviderHelper>>> = {
	...
	"ovhcloud": {
		"conversational": new OvhCloud.OvhCloudConversationalTask(),
	},
        ...

2 - Update packages/inference/src/providers/ovhcloud.ts to implement OvhCloudConversationalTask that inherits BaseConversationalTask:

import { BaseConversationalTask, BaseTextGenerationTask } from "./providerHelper";

export class OvhCloudConversationalTask extends BaseConversationalTask {
	constructor() {
		super("ovhcloud", "https://oai.endpoints.kepler.ai.cloud.ovh.net");
	}
}

and that's it :) let us know if you need any help! you can find more details in the documentation : https://huggingface.co/docs/inference-providers/register-as-a-provider#2-js-client-integration.

@julien-c
Copy link
Member

julien-c commented Apr 8, 2025

(sorry for the moving parts @fabienric – we can help move this PR over the finish line if needed)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
inference-providers integration of a new or existing Inference Provider
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants