Skip to content

Add v1 gRPC protocol crate#937

Open
bas3line wants to merge 2 commits into
HelixDB:mainfrom
bas3line:feature-grpc-protocol
Open

Add v1 gRPC protocol crate#937
bas3line wants to merge 2 commits into
HelixDB:mainfrom
bas3line:feature-grpc-protocol

Conversation

@bas3line

@bas3line bas3line commented Jun 13, 2026

Copy link
Copy Markdown

Description

Adds a dedicated helix-protocol workspace crate that defines the first versioned HelixDB gRPC contract and generates Rust bindings with Tonic/Prost.

This establishes the native gRPC surface requested in #780 without coupling the protocol contract to CLI or SDK internals. The server implementation can now implement helix.v1.Helix, and SDKs can generate clients from the same checked-in protobuf source.

Included API surface:

  • Query for unary dynamic/stored query execution.
  • QueryStream for large result sets and long-running traversals.
  • Insert for mutation-oriented document/record ingestion.
  • Search for streamed vector search results.
  • Health for cheap readiness checks.

The proto uses bytes for JSON payloads to preserve parity with the existing /v1/query HTTP contract while giving gRPC users a stable typed transport.

Related Issues

Addresses #780

Checklist when merging to main

  • No compiler warnings (if applicable)
  • Code is formatted with rustfmt
  • No useless or dead code (if applicable)
  • Code is easy to understand
  • Doc comments are used for all functions, enums, structs, and fields (where appropriate)
  • All tests pass
  • Performance has not regressed (assuming change was not to fix a bug)
  • Version number has been updated in helix-cli/Cargo.toml and helixdb/Cargo.toml

Additional Notes

Rust verification was run locally with cargo fmt --all, cargo test --workspace, and cargo clippy --workspace -- -D warnings.

The repo does not contain a literal helixdb/Cargo.toml; the version bump was applied to the helix-db crate in sdks/rust/Cargo.toml.

The crate uses protoc-bin-vendored in build.rs, so contributors and CI do not need a system protoc installation.

Greptile Summary

This PR introduces helix-protocol, a new workspace crate that defines the versioned helix.v1 gRPC contract using Tonic/Prost, and extends the Rust SDK's #[register] macro with additional parameter types and comprehensive documentation.

  • New helix-protocol crate: Adds helix/v1/helix.proto with five RPCs (Query, QueryStream, Insert, Search, Health), a build.rs that vendors protoc via protoc-bin-vendored, and a transport feature flag that gates tonic/transport for client/server flexibility.
  • Rust SDK macro improvements: helix-dsl-macros gains ParamObject, ParamValue, BTreeMap/HashMap map types, and improved recursive dynamic-value coercion with cleaner error paths.
  • Documentation pass: sdks/rust/src/lib.rs and sdks/rust/src/dsl.rs receive extensive doc-comment coverage including runnable examples and a new client_tests integration module.

Important Files Changed

Filename Overview
helix-protocol/proto/helix/v1/helix.proto New v1 proto contract; well-structured with reserved fields and cross-SDK options. Minor design concerns around SearchResult.sequence redundancy and non-optional id in MutationResponse.
helix-protocol/build.rs Uses std::env::set_var to inject protoc path; correct for protoc-bin-vendored with tonic-build 0.12, though the cleaner .protoc_path() builder method is available.
helix-protocol/src/lib.rs Clean protocol crate entry point; re-exports generated types under v1 module with DEFAULT_GRPC_PORT constant and three well-scoped tests for the generated message types.
helix-protocol/Cargo.toml Well-configured crate manifest; transport feature correctly gates tonic/transport, prost and tonic versions aligned at 0.13/0.12 respectively.
sdks/rust/helix-dsl-macros/src/lib.rs Comprehensive proc-macro for #[register]; new ParamObject/ParamValue type specs added with correct recursive dynamic-value coercion logic and good test coverage.
sdks/rust/src/lib.rs Extensive documentation additions and client_tests module added; logic unchanged, all existing API surfaces preserved.
Cargo.toml helix-protocol correctly added to workspace members list; no other changes.

Sequence Diagram

sequenceDiagram
    participant SDK as SDK / Client
    participant Tonic as helix-protocol<br/>(helix.v1.Helix)
    participant DB as HelixDB Server

    SDK->>Tonic: HelixClient::connect("http://localhost:6970")
    note over Tonic: gRPC over HTTP/2

    SDK->>DB: Query(QueryRequest)
    DB-->>SDK: "QueryResponse { json: bytes }"

    SDK->>DB: QueryStream(QueryRequest)
    loop stream
        DB-->>SDK: "QueryChunk { sequence, json }"
    end
    DB-->>SDK: END_STREAM

    SDK->>DB: Insert(InsertRequest)
    DB-->>SDK: "MutationResponse { id?, affected, json? }"

    SDK->>DB: Search(SearchRequest)
    loop stream (ranked)
        DB-->>SDK: "SearchResult { id, score, json, sequence }"
    end
    DB-->>SDK: END_STREAM

    SDK->>DB: "Health(HealthRequest {})"
    DB-->>SDK: "HealthResponse { status, version, message }"
Loading

Reviews (2): Last reviewed commit: "Address gRPC review feedback" | Re-trigger Greptile

Copilot AI review requested due to automatic review settings June 13, 2026 12:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds a new helix-protocol crate that defines HelixDB’s versioned Protobuf/gRPC contract and generates Rust bindings via tonic, then wires it into the workspace.

Changes:

  • Introduces helix.v1 Protobuf service/messages and Tonic-generated Rust types.
  • Adds a build script to compile protos with a vendored protoc.
  • Adds crate documentation and a few Rust unit tests for generated types.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
helix-protocol/src/lib.rs Exposes helix.v1 generated module + basic unit tests and a default port constant
helix-protocol/proto/helix/v1/helix.proto Defines the helix.v1.Helix gRPC service and request/response message schema
helix-protocol/build.rs Compiles the proto using vendored protoc and tonic-build
helix-protocol/README.md Documents API surface and shows Rust client usage
helix-protocol/Cargo.toml Adds crate metadata, dependencies, and build-dependencies for Tonic/Prost codegen
Cargo.toml Adds helix-protocol to the workspace members

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread helix-protocol/proto/helix/v1/helix.proto
Comment thread helix-protocol/proto/helix/v1/helix.proto
Comment thread helix-protocol/proto/helix/v1/helix.proto
Comment thread helix-protocol/proto/helix/v1/helix.proto
Comment thread helix-protocol/proto/helix/v1/helix.proto
@bas3line

Copy link
Copy Markdown
Author

@greptile-apps review the pr again lil bro

@xav-db

xav-db commented Jun 15, 2026

Copy link
Copy Markdown
Member

@matthewsanetra it would be good to go over this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants