Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feat][kubectl-plugin] Support node selectors for creating clusters #3143

Closed
MortalHappiness opened this issue Mar 3, 2025 · 5 comments · Fixed by #3228
Closed

[Feat][kubectl-plugin] Support node selectors for creating clusters #3143

MortalHappiness opened this issue Mar 3, 2025 · 5 comments · Fixed by #3228
Assignees
Labels
1.4.0 cli kubectl plugin P1 Issue that should be fixed within a few weeks

Comments

@MortalHappiness
Copy link
Member

Node selectors are very common to set for RayCluster, especially since most providers use node selectors as a way to specify the GPU type.

kubectl ray create cluster my-cluster \
  --node-selector=cloud.google.com/gke-accelerator=nvidia-l4,cloud.google.com/gke-nodepool=my-node-pool

Common node selectors like cloud.google.com/gke-accelerator and cloud.google.com/gke-spot can surface as provider-specific flags.

@MortalHappiness MortalHappiness added 1.4.0 cli kubectl plugin labels Mar 3, 2025
@kevin85421 kevin85421 added the P1 Issue that should be fixed within a few weeks label Mar 3, 2025
@EagleLo
Copy link

EagleLo commented Mar 9, 2025

Hi @MortalHappiness , I would like to take on this issue.

@kevin85421
Copy link
Member

@EagleLo what's your expected timeline for this item?

@EagleLo
Copy link

EagleLo commented Mar 14, 2025

@kevin85421 I originally scheduled to finish it by this weekend. Does that fit your expectation?

@kevin85421
Copy link
Member

Does that fit your expectation?

SG! Thanks

@EagleLo
Copy link

EagleLo commented Mar 17, 2025

Hi @kevin85421 and @MortalHappiness,

I've implemented the requested feature to support node selectors when creating Ray clusters with the kubectl plugin.

Changes Made

  • Added a new flag --node-selector to the kubectl ray create cluster command
  • Ensured node selectors are properly passed to both head and worker pod specifications

Testing

I've tested the implementation by first compiling the kubectl-ray plugin in the ./kubectl-plugin/cmd directory, which generated an executable file called "cmd". I then used this executable with the following command:

./cmd create cluster test-cluster --node-selector=cloud.google.com/gke-accelerator=nvidia-l4,cloud.google.com/gke-nodepool=my-node-pool --dry-run

The output is the following. The dry-run output shows that the node selectors are correctly applied to both the head and worker pod specifications:

apiVersion: ray.io/v1
kind: RayCluster
metadata:
  name: test-cluster
  namespace: default
spec:
  headGroupSpec:
    rayStartParams:
      dashboard-host: 0.0.0.0
    template:
      spec:
        containers:
        - image: rayproject/ray:2.41.0
          name: ray-head
          ports:
          - containerPort: 6379
            name: gcs-server
          - containerPort: 8265
            name: dashboard
          - containerPort: 10001
            name: client
          resources:
            limits:
              cpu: "2"
              memory: 4Gi
            requests:
              cpu: "2"
              memory: 4Gi
        **nodeSelector:
          cloud.google.com/gke-accelerator: nvidia-l4
          cloud.google.com/gke-nodepool: my-node-pool**
  rayVersion: 2.41.0
  workerGroupSpecs:
  - groupName: default-group
    rayStartParams:
      metrics-export-port: "8080"
    replicas: 1
    template:
      spec:
        containers:
        - image: rayproject/ray:2.41.0
          name: ray-worker
          resources:
            limits:
              cpu: "2"
              memory: 4Gi
            requests:
              cpu: "2"
              memory: 4Gi
        **nodeSelector:
          cloud.google.com/gke-accelerator: nvidia-l4
          cloud.google.com/gke-nodepool: my-node-pool**

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1.4.0 cli kubectl plugin P1 Issue that should be fixed within a few weeks
Projects
None yet
3 participants