fea: Registry API Data Source #2189

dmartinol · 2025-10-14T11:33:19Z

Fixes #2104

Changes

Added Registry API source handler with type: api
Added UT and IT
Updated docs

Postponed requirements

Management of upstream API and schema. Will track a separate issue for that

Signed-off-by: Daniele Martinoli <[email protected]>

codecov · 2025-10-14T11:58:21Z

Codecov Report

❌ Patch coverage is 83.51648% with 60 lines in your changes missing coverage. Please review.
✅ Project coverage is 53.47%. Comparing base (ffd3414) to head (b977f65).

Files with missing lines	Patch %	Lines
cmd/thv-operator/pkg/sources/api_toolhive.go	79.80%	31 Missing and 10 partials ⚠️
cmd/thv-operator/pkg/sources/api.go	79.22%	14 Missing and 2 partials ⚠️
cmd/thv-operator/pkg/httpclient/client.go	91.17%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2189      +/-   ##
==========================================
+ Coverage   53.09%   53.47%   +0.37%     
==========================================
  Files         222      227       +5     
  Lines       28904    29268     +364     
==========================================
+ Hits        15347    15650     +303     
- Misses      12419    12467      +48     
- Partials     1138     1151      +13

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Daniele Martinoli <[email protected]>

jhrozek · 2025-10-15T11:10:52Z

@claude provide a review. During the review, you must take into account the existing flows of the operator and how it synchronizes its data sources.

Copilot

Pull Request Overview

This PR adds support for fetching MCP registry data from API endpoints, enabling dynamic synchronization from remote registries via HTTP/HTTPS.

Key Changes:

Added API source handler with automatic format detection (ToolHive vs Upstream MCP Registry)
Implemented HTTP client for API operations with timeout and error handling
Added comprehensive unit and integration tests for API source functionality

Reviewed Changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`cmd/thv-operator/api/v1alpha1/mcpregistry_types.go`	Added `APISource` type and `RegistrySourceTypeAPI` constant to support API-based registry sources
`cmd/thv-operator/pkg/sources/api.go`	Main API source handler with format detection and delegation logic
`cmd/thv-operator/pkg/sources/api_toolhive.go`	ToolHive-specific API handler for fetching and converting registry data
`cmd/thv-operator/pkg/sources/api_upstream.go`	Upstream MCP Registry API handler (validation only, Phase 2 not implemented)
`cmd/thv-operator/pkg/sources/api_types.go`	Type aliases for ToolHive Registry API response types
`cmd/thv-operator/pkg/httpclient/client.go`	HTTP client implementation with context support and standard headers
`cmd/thv-operator/pkg/httpclient/types.go`	HTTPError type for structured error handling
`cmd/thv-operator/pkg/sources/factory.go`	Factory updated to create API source handlers
`deploy/charts/operator-crds/crds/toolhive.stacklok.dev_mcpregistries.yaml`	CRD updated with API source configuration
`docs/operator/crd-api.md`	Documentation for API source configuration
`cmd/thv-operator/REGISTRY.md`	User guide for API source configuration
`examples/operator/mcp-registries/*.yaml`	Example configurations for API source usage

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-15T11:11:05Z

cmd/thv-operator/api/v1alpha1/mcpregistry_types.go

+	//   - /v1/registry/servers - List all servers (single response, no pagination)
+	//   - /v1/registry/servers/{name} - Get specific server (future)
+	//   - /v1/registry/info - Get registry metadata (future)


The API paths in the documentation use /v1/registry/* but the actual implementation in api_toolhive.go uses /v0/* paths (e.g., /v0/info, /v0/servers). Update the documentation to match the implementation.

Suggested change

// - /v1/registry/servers - List all servers (single response, no pagination)

// - /v1/registry/servers/{name} - Get specific server (future)

// - /v1/registry/info - Get registry metadata (future)

// - /v0/servers - List all servers (single response, no pagination)

// - /v0/servers/{name} - Get specific server (future)

// - /v0/info - Get registry metadata (future)

Copilot · 2025-10-15T11:11:05Z

docs/operator/crd-api.md

+
+| Field | Description | Default | Validation |
+| --- | --- | --- | --- |
+| `endpoint` _string_ | Endpoint is the base API URL (without path)<br />The controller will append the appropriate paths:<br />Phase 1 (ToolHive API):<br />  - /v1/registry/servers - List all servers (single response, no pagination)<br />  - /v1/registry/servers/\{name\} - Get specific server (future)<br />  - /v1/registry/info - Get registry metadata (future)<br />Phase 2 (Upstream MCP Registry API):<br />  - /v0/servers - List all servers with pagination<br />  - /v0/servers/\{serverName\}/versions - List server versions<br />Example: "http://my-registry-api.default.svc.cluster.local/api" |  | MinLength: 1 <br />Pattern: `^https?://.*` <br />Required: \{\} <br /> |


The API paths in the documentation use /v1/registry/* for ToolHive API but the implementation uses /v0/* paths. Update the documentation to match the implementation's use of /v0/info, /v0/servers, etc.

Suggested change

| `endpoint` _string_ | Endpoint is the base API URL (without path) The controller will append the appropriate paths: Phase 1 (ToolHive API): - /v1/registry/servers - List all servers (single response, no pagination) - /v1/registry/servers/\{name\} - Get specific server (future) - /v1/registry/info - Get registry metadata (future) Phase 2 (Upstream MCP Registry API): - /v0/servers - List all servers with pagination - /v0/servers/\{serverName\}/versions - List server versions Example: "http://my-registry-api.default.svc.cluster.local/api" | | MinLength: 1 Pattern: `^https?://.*` Required: \{\} |

| `endpoint` _string_ | Endpoint is the base API URL (without path) The controller will append the appropriate paths: Phase 1 (ToolHive API): - /v0/servers - List all servers (single response, no pagination) - /v0/servers/\{name\} - Get specific server (future) - /v0/info - Get registry metadata (future) Phase 2 (Upstream MCP Registry API): - /v0/servers - List all servers with pagination - /v0/servers/\{serverName\}/versions - List server versions Example: "http://my-registry-api.default.svc.cluster.local/api" | | MinLength: 1 Pattern: `^https?://.*` Required: \{\} |

Copilot · 2025-10-15T11:11:05Z

deploy/charts/operator-crds/crds/toolhive.stacklok.dev_mcpregistries.yaml

+                          Phase 1 (ToolHive API):
+                            - /v1/registry/servers - List all servers (single response, no pagination)
+                            - /v1/registry/servers/{name} - Get specific server (future)
+                            - /v1/registry/info - Get registry metadata (future)
+                          Phase 2 (Upstream MCP Registry API):
+                            - /v0/servers - List all servers with pagination


The CRD documentation references /v1/registry/* paths for ToolHive API, but the implementation uses /v0/* paths. Update to match the actual implementation.

Suggested change

Phase 1 (ToolHive API):

- /v1/registry/servers - List all servers (single response, no pagination)

- /v1/registry/servers/{name} - Get specific server (future)

- /v1/registry/info - Get registry metadata (future)

Phase 2 (Upstream MCP Registry API):

- /v0/servers - List all servers with pagination

Supported API endpoints:

- /v0/servers - List all servers (with pagination)

Copilot · 2025-10-15T11:11:06Z

examples/operator/mcp-registries/mcpregistry-api-external.yaml

+    type: api
+    api:
+      # External registry API endpoint via HTTPS
+      # Base URL - controller appends /v0/servers


Comment says controller appends /v0/servers which is correct according to the implementation. However, the API type documentation incorrectly states /v1/registry/servers. This comment is accurate - no change needed here, but confirms the documentation issue elsewhere.

claude · 2025-10-15T11:11:07Z

Claude encountered an error —— View job

Failed with exit code 128

I'll analyze this and get back to you.

dmartinol and others added 5 commits October 13, 2025 14:36

impl draft

9850663

Signed-off-by: Daniele Martinoli <[email protected]>

Registry API Source Handler

3bb4a84

Signed-off-by: Daniele Martinoli <[email protected]>

Merge branch 'stacklok:main' into registry_apisource

bbb6e0c

chart version bump

a3958b9

Signed-off-by: Daniele Martinoli <[email protected]>

Merge branch 'stacklok:main' into registry_apisource

61d24de

dmartinol and others added 2 commits October 14, 2025 14:56

increase test coverage

790debb

Signed-off-by: Daniele Martinoli <[email protected]>

Merge branch 'stacklok:main' into registry_apisource

b977f65

jhrozek requested a review from Copilot October 15, 2025 11:09

Copilot AI reviewed Oct 15, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fea: Registry API Data Source #2189

fea: Registry API Data Source #2189

Uh oh!

dmartinol commented Oct 14, 2025

Uh oh!

codecov bot commented Oct 14, 2025 •

edited

Loading

Uh oh!

jhrozek commented Oct 15, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Oct 15, 2025

Uh oh!

Copilot AI Oct 15, 2025

Uh oh!

Copilot AI Oct 15, 2025

Uh oh!

Copilot AI Oct 15, 2025

Uh oh!

claude bot commented Oct 15, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

fea: Registry API Data Source #2189

Are you sure you want to change the base?

fea: Registry API Data Source #2189

Uh oh!

Conversation

dmartinol commented Oct 14, 2025

Changes

Postponed requirements

Uh oh!

codecov bot commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jhrozek commented Oct 15, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

claude bot commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov bot commented Oct 14, 2025 •

edited

Loading

claude bot commented Oct 15, 2025 •

edited

Loading