Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for weight based queue management in dynamiccontroller #198

Open
a-hilaly opened this issue Jan 11, 2025 · 0 comments
Open

Add support for weight based queue management in dynamiccontroller #198

a-hilaly opened this issue Jan 11, 2025 · 0 comments

Comments

@a-hilaly
Copy link
Member

Feature Description

Currently, the DynamicController uses a single queue for all GVRs, which doesn't allow for prioritization of resource processing based on user requirements. This can lead to resource starvation and unpredictable processing times for important resources.

One of the optimisations we can work on is a weight based queue system where resources are processed according to weights specified in ResourceGroup specs. Instead of maintaining a queue per GVR, we'll maintain queues per unique weight value.

We can add a spec.weight field to the ResourceGroup CRD to allow users to specify the processing priority of their resources.

Thoughts: the weight will be an integer value between 1 and 1000, with a default value of 100 if not specified. Resources with higher weights will receive proportionally more processing time and workers from the controller.

Sketching:

type WeightedQueue struct {
    weight    int
    queue     workqueue.RateLimitingInterface 
    gvrSet    map[schema.GroupVersionResource]struct{}
}

type DynamicController struct {
    weightedQueues map[int]*WeightedQueue
    gvrWeights     map[schema.GroupVersionResource]int
    mu             sync.RWMutex
}
@a-hilaly a-hilaly changed the title Add Support for weight based queue management in dynamiccontroller Add support for weight based queue management in dynamiccontroller Jan 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant