Skip to content

Conversation

dekaihu
Copy link

@dekaihu dekaihu commented Sep 28, 2025

What type of PR is this?

/kind feature

What this PR does / why we need it:

Which issue(s) this PR fixes:

Part of #6589

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

`karmada-controller-manager`: Introduced built-in interpreter for Volcano `Job`.

@karmada-bot
Copy link
Collaborator

Welcome @dekaihu! It looks like this is your first PR to karmada-io/karmada 🎉

Copy link

Summary of Changes

Hello @dekaihu, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request integrates support for Volcano Jobs (batch.volcano.sh/v1alpha1/Job) by adding a dedicated resource interpreter. This interpreter equips the system with the capability to manage and synchronize Volcano Jobs across multiple clusters, providing essential functions for replica management, health monitoring, resource allocation, and comprehensive status aggregation and reflection. This enhancement significantly broadens the types of batch workloads that can be effectively orchestrated and observed.

Highlights

  • New Volcano Job Interpreter: This pull request introduces a new ResourceInterpreterCustomization specifically for batch.volcano.sh/v1alpha1/Job resources, enabling a multi-cluster management system to understand and manage Volcano Jobs.
  • Comprehensive Interpretation Logic: Lua scripts are implemented for various interpretation functions, including replicaRevision to adjust minAvailable, healthInterpretation to determine job health based on phase, and componentResource to extract resource requirements from job tasks.
  • Advanced Status Management: Detailed logic is provided for statusAggregation to combine status information from multiple clusters and statusReflection to accurately reflect the complex status of Volcano Jobs, including task status counts and conditions.
  • Test Coverage: Accompanying test configurations and sample status data are added to validate the functionality of the new Volcano Job interpreter across different operations like status aggregation, replica interpretation, and health checks.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@karmada-bot karmada-bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Sep 28, 2025
Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a resource interpreter for Volcano jobs. The implementation is a good start, but I've found several issues, primarily within the Lua scripts for status aggregation and component resource discovery. There are cases where values are overwritten instead of being aggregated, which would lead to incorrect behavior. I've also identified a critical typo that would cause a runtime error. My review includes specific suggestions to address these points and improve the robustness of the interpreter.

@dekaihu dekaihu force-pushed the interpreter-vcjob branch 3 times, most recently from 11d31f9 to 0876da6 Compare September 29, 2025 02:08
@codecov-commenter
Copy link

codecov-commenter commented Sep 29, 2025

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 45.84%. Comparing base (4594ca0) to head (bc07584).
⚠️ Report is 42 commits behind head on master.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6790      +/-   ##
==========================================
+ Coverage   45.73%   45.84%   +0.11%     
==========================================
  Files         689      690       +1     
  Lines       57104    57300     +196     
==========================================
+ Hits        26114    26269     +155     
- Misses      29358    29401      +43     
+ Partials     1632     1630       -2     
Flag Coverage Δ
unittests 45.84% <ø> (+0.11%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Member

@RainbowMango RainbowMango left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/assign

@RainbowMango
Copy link
Member

/gemini review

Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for Volcano Job resource interpretation in Karmada by implementing a ResourceInterpreterCustomization for batch.volcano.sh/v1alpha1/Job resources.

  • Implements comprehensive resource interpretation including replica revision, health checks, component resource analysis, and status aggregation/reflection
  • Adds test infrastructure with test configuration and status test data
  • Provides Lua scripts for handling Volcano Job lifecycle operations in multi-cluster environments

Reviewed Changes

Copilot reviewed 3 out of 5 changed files in this pull request and generated 2 comments.

File Description
customizations.yaml Main ResourceInterpreterCustomization with Lua scripts for Volcano Job interpretation
customizations_tests.yaml Test configuration file (currently commented out) for validating the interpreter
status-file.yaml Test data representing aggregated status information for testing status operations

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a resource interpreter for VolcanoJob (batch.volcano.sh/v1alpha1.Job). The implementation covers replica revision, health interpretation, component resource discovery, status aggregation, and status reflection using Lua scripts.

My review has identified a few issues:

  • A potential runtime error in the InterpretHealth function.
  • A bug in the durationVal function within statusAggregation that would lead to incorrect duration comparisons.
  • Some redundant code in the statusReflection function.
  • The tests for the interpreter are currently commented out, and required test data files are missing, which prevents verification of the implementation.

I've provided specific comments and suggestions to address these points. Please enable and complete the tests to ensure the interpreter is robust.

@karmada-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from rainbowmango. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@dekaihu dekaihu force-pushed the interpreter-vcjob branch 5 times, most recently from fadcbf7 to 4963cf3 Compare September 29, 2025 11:05
@dekaihu
Copy link
Author

dekaihu commented Sep 30, 2025

Test Steps:

  1. Enable the MultiplePodTemplatesScheduling feature in Karmada 1.15.
  2. Use Karmada to deploy the VolcanoJob interpreter and VolcanoJob CRD.
  3. Distribute the VolcanoJob on the Karmada cluster.
    Results:
    VolcanoJob template reference:
apiVersion: batch.volcano.sh/v1alpha1
kind: Job
metadata:
  name: dk-job
spec:
  maxRetry: 3
  minAvailable: 3
  plugins:
    env: []
    ssh: []
    svc:
    - --disable-network-policy=true
  queue: default
  schedulerName: volcano
  tasks:
  - minAvailable: 1
    name: job-nginx1
    replicas: 1
    template:
      metadata:
        name: nginx1
      spec:
        containers:
        - args:
          - sleep 10
          command:
          - bash
          - -c
          image: nginx:latest
          imagePullPolicy: IfNotPresent
          name: nginx
          resources:
            requests:
              cpu: 100m
        nodeSelector:
          kubernetes.io/os: linux
        restartPolicy: OnFailure
  - minAvailable: 2
    name: job-nginx2
    replicas: 3
    template:
      metadata:
        name: nginx2
      spec:
        containers:
        - args:
          - sleep 30
          command:
          - bash
          - -c
          image: nginx:latest
          imagePullPolicy: IfNotPresent
          name: nginx
          resources:
            requests:
              cpu: 100m
        nodeSelector:
          kubernetes.io/os: linux
        restartPolicy: OnFailure
  1. Single-member cluster scheduling
    View resource requests in the resourcebinding.
  clusters:
  - name: member2
  components:
  - name: job-nginx1
    replicaRequirements:
      nodeClaim:
        nodeSelector:
          kubernetes.io/os: linux
      resourceRequest:
        cpu: 100m
    replicas: 1
  - name: job-nginx2
    replicaRequirements:
      nodeClaim:
        nodeSelector:
          kubernetes.io/os: linux
      resourceRequest:
        cpu: 100m
    replicas: 2

The results on the Karmada control plane are as follows:

status:
  conditions:
  - lastTransitionTime: "2025-09-29T10:57:03Z"
    status: Pending
  - lastTransitionTime: "2025-09-29T10:57:07Z"
    status: Running
  - lastTransitionTime: "2025-09-29T10:57:38Z"
    status: Completed
  minAvailable: 3
  runningDuration: 35.688396584s
  state:
    lastTransitionTime: "2025-09-29T10:57:38Z"
    phase: Completed
  succeeded: 4
  taskStatusCount:
    job-nginx1:
      phase:
        Succeeded: 1
    job-nginx2:
      phase:
        Succeeded: 3
  version: 1

Results on the member cluster control plane:

status:
  conditions:
  - lastTransitionTime: "2025-09-29T10:57:03Z"
    status: Pending
  - lastTransitionTime: "2025-09-29T10:57:07Z"
    status: Running
  - lastTransitionTime: "2025-09-29T10:57:38Z"
    status: Completed
  minAvailable: 3
  runningDuration: 35.688396584s
  state:
    lastTransitionTime: "2025-09-29T10:57:38Z"
    phase: Completed
  succeeded: 4
  taskStatusCount:
    job-nginx1:
      phase:
        Succeeded: 1
    job-nginx2:
      phase:
        Succeeded: 3
  version: 1
  1. Aggregated results for multi-cluster scheduling
    View resource requests in the resourcebinding.
spec:
  clusters:
  - name: member1
  - name: member2
  components:
  - name: job-nginx1
    replicaRequirements:
      nodeClaim:
        nodeSelector:
          kubernetes.io/os: linux
      resourceRequest:
        cpu: 100m
    replicas: 1
  - name: job-nginx2
    replicaRequirements:
      nodeClaim:
        nodeSelector:
          kubernetes.io/os: linux
      resourceRequest:
        cpu: 100m
    replicas: 2

The results on the Karmada control plane are as follows:

status:
  conditions:
  - lastTransitionTime: "2025-09-30T02:07:56Z"
    status: Pending
  - lastTransitionTime: "2025-09-30T02:08:08Z"
    status: Running
  - lastTransitionTime: "2025-09-30T02:08:40Z"
    status: Completed
  - lastTransitionTime: "2025-09-30T02:07:56Z"
    status: Pending
  - lastTransitionTime: "2025-09-30T02:08:02Z"
    status: Running
  - lastTransitionTime: "2025-09-30T02:08:34Z"
    status: Completed
  minAvailable: 6
  runningDuration: 44.585525557s
  state:
    lastTransitionTime: "2025-09-30T02:08:34Z"
    phase: Completed
  succeeded: 8
  taskStatusCount:
    job-nginx1:
      phase:
        Succeeded: 2
    job-nginx2:
      phase:
        Succeeded: 6
  version: 1

Results on the member cluster control plane:
member1:

status:
  conditions:
  - lastTransitionTime: "2025-09-30T02:07:56Z"
    status: Pending
  - lastTransitionTime: "2025-09-30T02:08:08Z"
    status: Running
  - lastTransitionTime: "2025-09-30T02:08:40Z"
    status: Completed
  minAvailable: 3
  runningDuration: 44.585525557s
  state:
    lastTransitionTime: "2025-09-30T02:08:40Z"
    phase: Completed
  succeeded: 4
  taskStatusCount:
    job-nginx1:
      phase:
        Succeeded: 1
    job-nginx2:
      phase:
        Succeeded: 3
  version: 1

member2:

status:
  conditions:
  - lastTransitionTime: "2025-09-30T02:07:56Z"
    status: Pending
  - lastTransitionTime: "2025-09-30T02:08:02Z"
    status: Running
  - lastTransitionTime: "2025-09-30T02:08:34Z"
    status: Completed
  minAvailable: 3
  runningDuration: 38.49844198s
  state:
    lastTransitionTime: "2025-09-30T02:08:34Z"
    phase: Completed
  succeeded: 4
  taskStatusCount:
    job-nginx1:
      phase:
        Succeeded: 1
    job-nginx2:
      phase:
        Succeeded: 3
  version: 1

status.unknown = status.unknown + (s.unknown or 0)
status.version = math.max(status.version, s.version or 0)
status.retryCount = status.retryCount + (s.retryCount or 0)
status.state = s.state
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The state might be overwritten unexpectly, for instance, we are aggreatingstates from two member clusters, the second state would overrite the first one.
But it good enough for current stage, as Karmada not supports split Volcano Jobs currently.

It's hard to aggreate the state, we have met this situation when aggreating Kubernetes Jobs at https://github.com/karmada-io/karmada/blob/adef1e59748e1e1d31cb75fffe406b5dd69a66d7/pkg/util/helper/job.go#L76-L9.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dekaihu Any idea about how to handle the state aggregation?

@dekaihu dekaihu force-pushed the interpreter-vcjob branch from b39d0bd to abdb0c4 Compare October 9, 2025 10:51
@dekaihu dekaihu requested a review from RainbowMango October 9, 2025 12:02
Signed-off-by: hudekai <[email protected]>
@dekaihu dekaihu force-pushed the interpreter-vcjob branch from 53fff15 to bc07584 Compare October 9, 2025 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants