Skip to content

Commit 1797ed9

Browse files
UriZafrirurizaf-workonematchfoxEItanyapeterj
authored
feat: implement backend for models ui (#690)
* feat: implement backend for models ui Signed-off-by: urizaf-work <uri.zafrir@kaltura.com> Signed-off-by: urizaf <urizaf@gmail.com> * add missing providers so they all appear in ui Signed-off-by: urizaf-work <uri.zafrir@kaltura.com> Signed-off-by: urizaf <urizaf@gmail.com> * fix(ui): correctly display args for tool calls in chat (#688) Signed-off-by: Brian Fox <878612+onematchfox@users.noreply.github.com> Signed-off-by: urizaf <urizaf@gmail.com> * update READMEs based on new architecture (#684) * update READMEs based on new architecture Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io> * PR comments Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io> * Update ui/README.md --------- Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io> Co-authored-by: Peter Jausovec <peterj@users.noreply.github.com> Signed-off-by: urizaf <urizaf@gmail.com> * [FIX ] - fixes adk performance tuning (#689) * - fixes adk performance tuning - dependency versions update Signed-off-by: Dmytro Rashko <dmitriy.rashko@amdocs.com> * fix VERSION in case forked repository without tags Signed-off-by: Dmytro Rashko <dmitriy.rashko@amdocs.com> * revert uv version Signed-off-by: Dmytro Rashko <dmitriy.rashko@amdocs.com> * updated golden e2e Signed-off-by: Dmytro Rashko <dmitriy.rashko@amdocs.com> * fix helm unit tests Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io> --------- Signed-off-by: Dmytro Rashko <dmitriy.rashko@amdocs.com> Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io> Co-authored-by: Eitan Yarmush <eitan.yarmush@solo.io> Signed-off-by: urizaf <urizaf@gmail.com> * feat: make streaming buffer size configurable (#696) * feat: make streaming buffer size configurable Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io> * switch to resource quantities for buffer size Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io> --------- Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io> Signed-off-by: urizaf <urizaf@gmail.com> * eitanya/fix-python-release (#698) Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io> Signed-off-by: urizaf <urizaf@gmail.com> * EP-685-kmcp (#686) * EP-685-kmcp Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io> * Update design/EP-685-kmcp.md Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io> --------- Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io> Co-authored-by: Lin Sun <lin.sun@solo.io> Signed-off-by: urizaf <urizaf@gmail.com> * use the types defined in pkg/client/model.go Signed-off-by: urizaf <urizaf@gmail.com> * fix(ui): correct link to switch agent from within chat (#667) Signed-off-by: Brian Fox <878612+onematchfox@users.noreply.github.com> Co-authored-by: Peter Jausovec <peterj@users.noreply.github.com> Signed-off-by: urizaf <urizaf@gmail.com> * fix(ui): display description for agent tools when editing an agent (#692) Signed-off-by: Brian Fox <878612+onematchfox@users.noreply.github.com> Co-authored-by: Peter Jausovec <peterj@users.noreply.github.com> Signed-off-by: urizaf <urizaf@gmail.com> * fix(controller): watch secondary resources instead of updating unowned resources (#703) * fix(controller): watch secrets from agents controller Ref: https://book.kubebuilder.io/reference/watching-resources/secondary-resources-not-owned Signed-off-by: Brian Fox <878612+onematchfox@users.noreply.github.com> * fix(controller): watch memory from agents controller Signed-off-by: Brian Fox <878612+onematchfox@users.noreply.github.com> * fix(controller): watch toolservers from agents controller Signed-off-by: Brian Fox <878612+onematchfox@users.noreply.github.com> * fix(controller): watch modelconfig from agent controller Signed-off-by: Brian Fox <878612+onematchfox@users.noreply.github.com> * fix(controller): watch secrets from model config controller Agent watches ModelConfig -> ModelConfig watches Secret Signed-off-by: Brian Fox <878612+onematchfox@users.noreply.github.com> * refactor(controller): consistent error logging Signed-off-by: Brian Fox <878612+onematchfox@users.noreply.github.com> * refactor(controller): remove `reconcileAgents` This isn't needed any more - we only ever reconcile a single agent at a time now. Signed-off-by: Brian Fox <878612+onematchfox@users.noreply.github.com> * fix(controller): ensure api key secret exists for model config Signed-off-by: Brian Fox <878612+onematchfox@users.noreply.github.com> * refactor(controller): explicitly set error to nil for memory status Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Brian Fox <878612+onematchfox@users.noreply.github.com> --------- Signed-off-by: Brian Fox <878612+onematchfox@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: urizaf <urizaf@gmail.com> * change to correct Gemini icon Signed-off-by: urizaf <urizaf@gmail.com> --------- Signed-off-by: urizaf-work <uri.zafrir@kaltura.com> Signed-off-by: urizaf <urizaf@gmail.com> Signed-off-by: Brian Fox <878612+onematchfox@users.noreply.github.com> Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io> Signed-off-by: Dmytro Rashko <dmitriy.rashko@amdocs.com> Co-authored-by: urizaf-work <uri.zafrir@kaltura.com> Co-authored-by: Brian Fox <878612+onematchfox@users.noreply.github.com> Co-authored-by: Eitan Yarmush <eitan.yarmush@solo.io> Co-authored-by: Peter Jausovec <peterj@users.noreply.github.com> Co-authored-by: Dmytro Rashko <dmitriy.rashko@amdocs.com> Co-authored-by: Lin Sun <lin.sun@solo.io> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent 9160376 commit 1797ed9

File tree

8 files changed

+150
-21
lines changed

8 files changed

+150
-21
lines changed

‎go/internal/httpserver/handlers/models.go‎

Lines changed: 47 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,12 @@ package handlers
33
import (
44
"net/http"
55

6-
"github.com/kagent-dev/kagent/go/internal/httpserver/errors"
6+
"github.com/kagent-dev/kagent/go/pkg/client/api"
7+
kclient "github.com/kagent-dev/kagent/go/pkg/client"
78
ctrllog "sigs.k8s.io/controller-runtime/pkg/log"
89
)
910

11+
1012
// ModelHandler handles model requests
1113
type ModelHandler struct {
1214
*Base
@@ -22,8 +24,49 @@ func (h *ModelHandler) HandleListSupportedModels(w ErrorResponseWriter, r *http.
2224

2325
log.Info("Listing supported models")
2426

25-
// TODO: Implement this
27+
// Create a map of provider names to their supported models
28+
// The keys need to match what the UI expects (camelCase for API keys)
29+
supportedModels := kclient.ProviderModels{
30+
"openAI": {
31+
{Name: "gpt-4o", FunctionCalling: true},
32+
{Name: "gpt-4-turbo", FunctionCalling: true},
33+
{Name: "gpt-4", FunctionCalling: true},
34+
{Name: "gpt-3.5-turbo", FunctionCalling: true},
35+
},
36+
"anthropic": {
37+
{Name: "claude-3-opus-20240229", FunctionCalling: true},
38+
{Name: "claude-3-sonnet-20240229", FunctionCalling: true},
39+
{Name: "claude-3-haiku-20240307", FunctionCalling: true},
40+
{Name: "claude-2.1", FunctionCalling: false},
41+
{Name: "claude-2.0", FunctionCalling: false},
42+
},
43+
"azureOpenAI": {
44+
{Name: "gpt-4", FunctionCalling: true},
45+
{Name: "gpt-35-turbo", FunctionCalling: true},
46+
},
47+
"ollama": {
48+
{Name: "llama2", FunctionCalling: false},
49+
{Name: "llama2:13b", FunctionCalling: false},
50+
{Name: "llama2:70b", FunctionCalling: false},
51+
{Name: "mistral", FunctionCalling: false},
52+
{Name: "mixtral", FunctionCalling: false},
53+
},
54+
"gemini": {
55+
{Name: "gemini-pro", FunctionCalling: true},
56+
{Name: "gemini-pro-vision", FunctionCalling: false},
57+
},
58+
"geminiVertexAI": {
59+
{Name: "gemini-pro", FunctionCalling: true},
60+
{Name: "gemini-pro-vision", FunctionCalling: false},
61+
},
62+
"anthropicVertexAI": {
63+
{Name: "claude-3-opus-20240229", FunctionCalling: true},
64+
{Name: "claude-3-sonnet-20240229", FunctionCalling: true},
65+
{Name: "claude-3-haiku-20240307", FunctionCalling: true},
66+
},
67+
}
2668

27-
w.RespondWithError(errors.NewNotImplementedError("Not implemented", nil))
28-
return
69+
log.Info("Successfully listed supported models", "count", len(supportedModels))
70+
data := api.NewResponse(supportedModels, "Successfully listed supported models", false)
71+
RespondWithJSON(w, http.StatusOK, data)
2972
}

‎go/internal/httpserver/handlers/providers.go‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,9 @@ func (h *ProviderHandler) HandleListSupportedModelProviders(w ErrorResponseWrite
9797
{v1alpha1.ModelProviderAnthropic, reflect.TypeOf(v1alpha1.AnthropicConfig{})},
9898
{v1alpha1.ModelProviderAzureOpenAI, reflect.TypeOf(v1alpha1.AzureOpenAIConfig{})},
9999
{v1alpha1.ModelProviderOllama, reflect.TypeOf(v1alpha1.OllamaConfig{})},
100+
{v1alpha1.ModelProviderGemini, reflect.TypeOf(v1alpha1.GeminiConfig{})},
101+
{v1alpha1.ModelProviderGeminiVertexAI, reflect.TypeOf(v1alpha1.GeminiVertexAIConfig{})},
102+
{v1alpha1.ModelProviderAnthropicVertexAI, reflect.TypeOf(v1alpha1.AnthropicVertexAIConfig{})},
100103
}
101104

102105
providersResponse := []map[string]interface{}{}

‎go/pkg/client/model.go‎

Lines changed: 29 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,23 @@
11
package client
22

3+
import (
4+
"context"
5+
6+
"github.com/kagent-dev/kagent/go/pkg/client/api"
7+
)
8+
9+
// ModelInfo represents information about a model
10+
type ModelInfo struct {
11+
Name string `json:"name"`
12+
FunctionCalling bool `json:"function_calling"`
13+
}
14+
15+
// ProviderModels represents a map of provider names to their supported models
16+
type ProviderModels map[string][]ModelInfo
17+
318
// Model defines the model operations
419
type Model interface {
5-
// ListSupportedModels(ctx context.Context) (*api.StandardResponse[*client.ProviderModels], error)
20+
ListSupportedModels(ctx context.Context) (*api.StandardResponse[ProviderModels], error)
621
}
722

823
// modelClient handles model-related requests
@@ -16,16 +31,16 @@ func NewModelClient(client *BaseClient) Model {
1631
}
1732

1833
// ListSupportedModels lists all supported models
19-
// func (c *modelClient) ListSupportedModels(ctx context.Context) (*api.StandardResponse[*client.ProviderModels], error) {
20-
// resp, err := c.client.Get(ctx, "/api/models", "")
21-
// if err != nil {
22-
// return nil, err
23-
// }
24-
25-
// var models api.StandardResponse[*client.ProviderModels]
26-
// if err := DecodeResponse(resp, &models); err != nil {
27-
// return nil, err
28-
// }
29-
30-
// return &models, nil
31-
// }
34+
func (c *modelClient) ListSupportedModels(ctx context.Context) (*api.StandardResponse[ProviderModels], error) {
35+
resp, err := c.client.Get(ctx, "/api/models", "")
36+
if err != nil {
37+
return nil, err
38+
}
39+
40+
var models api.StandardResponse[ProviderModels]
41+
if err := DecodeResponse(resp, &models); err != nil {
42+
return nil, err
43+
}
44+
45+
return &models, nil
46+
}

‎ui/src/components/ModelProviderCombobox.tsx‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import { OpenAI } from './icons/OpenAI';
1010
import { Anthropic } from './icons/Anthropic';
1111
import { Ollama } from './icons/Ollama';
1212
import { Azure } from './icons/Azure';
13+
import { Gemini } from './icons/Gemini';
1314

1415
interface ComboboxOption {
1516
label: string; // e.g., "OpenAI - gpt-4o"
@@ -59,6 +60,9 @@ export function ModelProviderCombobox({
5960
'anthropic': Anthropic,
6061
'ollama': Ollama,
6162
'azure-openai': Azure,
63+
'gemini': Gemini,
64+
'gemini-vertex-ai': Gemini,
65+
'anthropic-vertex-ai': Anthropic,
6266
};
6367

6468
const getProviderIcon = (providerKey: ModelProviderKey | undefined): React.ReactNode | null => {

‎ui/src/components/icons/Gemini.tsx‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
export function Gemini({ className }: { className?: string }) {
2+
return (
3+
<svg fill="none" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16" className={className}><path d="M16 8.016A8.522 8.522 0 008.016 16h-.032A8.521 8.521 0 000 8.016v-.032A8.521 8.521 0 007.984 0h.032A8.522 8.522 0 0016 7.984v.032z" fill="url(#prefix__paint0_radial_980_20147)"/><defs><radialGradient id="prefix__paint0_radial_980_20147" cx="0" cy="0" r="1" gradientUnits="userSpaceOnUse" gradientTransform="matrix(16.1326 5.4553 -43.70045 129.2322 1.588 6.503)"><stop offset=".067" stop-color="#9168C0"/><stop offset=".343" stop-color="#5684D1"/><stop offset=".672" stop-color="#1BA1E3"/></radialGradient></defs></svg>
4+
);
5+
}

‎ui/src/components/onboarding/steps/ModelConfigStep.tsx‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ import { k8sRefUtils } from '@/lib/k8sUtils';
2626
import { K8S_AGENT_DEFAULTS } from '../OnboardingWizard';
2727
import { NamespaceCombobox } from "@/components/NamespaceCombobox";
2828

29-
const modelProviders = ["openai", "azure-openai", "anthropic", "ollama"] as const;
29+
const modelProviders = ["openai", "azure-openai", "anthropic", "ollama", "gemini", "gemini-vertex-ai", "anthropic-vertex-ai"] as const;
3030
const modelConfigSchema = z.object({
3131
providerName: z.enum(modelProviders, { required_error: "Please select a provider." }),
3232
configName: z.string().min(1, "Configuration name is required."),
@@ -190,6 +190,9 @@ export function ModelConfigStep({
190190
payload.azureOpenAI = { azureEndpoint: values.azureEndpoint || "", apiVersion: values.azureApiVersion || "" }; break;
191191
case 'openai': payload.openAI = {}; break;
192192
case 'anthropic': payload.anthropic = {}; break;
193+
case 'gemini': payload.gemini = {}; break;
194+
case 'gemini-vertex-ai': payload.geminiVertexAI = {}; break;
195+
case 'anthropic-vertex-ai': payload.anthropicVertexAI = {}; break;
193196
case 'ollama':
194197
const modelTag = values.modelTag?.trim() || "";
195198
if (modelTag && modelTag !== OLLAMA_DEFAULT_TAG) {

‎ui/src/lib/providers.ts‎

Lines changed: 26 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11

2-
export type BackendModelProviderType = "OpenAI" | "AzureOpenAI" | "Anthropic" | "Ollama";
3-
export const modelProviders = ["openai", "azure-openai", "anthropic", "ollama"] as const;
2+
export type BackendModelProviderType = "OpenAI" | "AzureOpenAI" | "Anthropic" | "Ollama" | "Gemini" | "GeminiVertexAI" | "AnthropicVertexAI";
3+
export const modelProviders = ["openai", "azure-openai", "anthropic", "ollama", "gemini", "gemini-vertex-ai", "anthropic-vertex-ai"] as const;
44
export type ModelProviderKey = typeof modelProviders[number];
55

66

@@ -41,6 +41,27 @@ export const PROVIDERS_INFO: {
4141
modelDocsLink: "https://github.com/kagent-dev/autogen/blob/main/python/packages/autogen-ext/src/autogen_ext/models/ollama/_model_info.py",
4242
help: "No API key needed. Ensure Ollama is running and accessible."
4343
},
44+
gemini: {
45+
name: "Gemini",
46+
type: "Gemini",
47+
apiKeyLink: "https://ai.google.dev/",
48+
modelDocsLink: "https://ai.google.dev/docs",
49+
help: "Get your API key from the Google AI Studio."
50+
},
51+
"gemini-vertex-ai": {
52+
name: "Gemini Vertex AI",
53+
type: "GeminiVertexAI",
54+
apiKeyLink: "https://cloud.google.com/vertex-ai",
55+
modelDocsLink: "https://cloud.google.com/vertex-ai/docs",
56+
help: "Configure your Google Cloud project and credentials for Vertex AI."
57+
},
58+
"anthropic-vertex-ai": {
59+
name: "Anthropic Vertex AI",
60+
type: "AnthropicVertexAI",
61+
apiKeyLink: "https://cloud.google.com/vertex-ai",
62+
modelDocsLink: "https://cloud.google.com/vertex-ai/docs",
63+
help: "Configure your Google Cloud project and credentials for Vertex AI."
64+
},
4465
};
4566

4667
export const isValidProviderInfoKey = (key: string): key is ModelProviderKey => {
@@ -54,6 +75,9 @@ export const getApiKeyForProviderFormKey = (providerFormKey: ModelProviderKey):
5475
case 'azure-openai': return 'azureOpenAI';
5576
case 'anthropic': return 'anthropic';
5677
case 'ollama': return 'ollama';
78+
case 'gemini': return 'gemini';
79+
case 'gemini-vertex-ai': return 'geminiVertexAI';
80+
case 'anthropic-vertex-ai': return 'anthropicVertexAI';
5781
default: return providerFormKey;
5882
}
5983
};

‎ui/src/types/index.ts‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,32 @@ export interface OllamaConfigPayload {
8181
options?: Record<string, string>;
8282
}
8383

84+
export interface GeminiConfigPayload {
85+
baseUrl?: string;
86+
temperature?: string;
87+
maxTokens?: number;
88+
topP?: string;
89+
topK?: number;
90+
}
91+
92+
export interface GeminiVertexAIConfigPayload {
93+
project?: string;
94+
location?: string;
95+
temperature?: string;
96+
maxTokens?: number;
97+
topP?: string;
98+
topK?: number;
99+
}
100+
101+
export interface AnthropicVertexAIConfigPayload {
102+
project?: string;
103+
location?: string;
104+
temperature?: string;
105+
maxTokens?: number;
106+
topP?: string;
107+
topK?: number;
108+
}
109+
84110
export interface CreateModelConfigPayload {
85111
ref: string;
86112
provider: Pick<Provider, "name" | "type">;
@@ -90,6 +116,9 @@ export interface CreateModelConfigPayload {
90116
anthropic?: AnthropicConfigPayload;
91117
azureOpenAI?: AzureOpenAIConfigPayload;
92118
ollama?: OllamaConfigPayload;
119+
gemini?: GeminiConfigPayload;
120+
geminiVertexAI?: GeminiVertexAIConfigPayload;
121+
anthropicVertexAI?: AnthropicVertexAIConfigPayload;
93122
}
94123

95124
export interface UpdateModelConfigPayload {
@@ -100,6 +129,9 @@ export interface UpdateModelConfigPayload {
100129
anthropic?: AnthropicConfigPayload;
101130
azureOpenAI?: AzureOpenAIConfigPayload;
102131
ollama?: OllamaConfigPayload;
132+
gemini?: GeminiConfigPayload;
133+
geminiVertexAI?: GeminiVertexAIConfigPayload;
134+
anthropicVertexAI?: AnthropicVertexAIConfigPayload;
103135
}
104136

105137
export interface MemoryResponse {

0 commit comments

Comments
 (0)