karmada-scheduler could support an incremental scheduling capability in divided replicas strategy

[LivingCcj]

What would you like to be added:
Add a new replicas scheduling scenarios In divided replicas scheduling type, which could support an incremental delivery replicas strategy. If the available replicas in member-clusters is less than the target replicas, the karmada-scheduler should deliver the available replicas to member-clusters preferentially.

Why is this needed:
In Divided replicas scheduling type, user want to scale up by 100 pods, but the member clusters could place 95 pods. however the scheduler will meet err not enough to schedule.

In the cases, that expects karmada-scheduler to divide the 95 pods into member cluster preferentially, and then keep going to divide the remaining pods.

In emergency workload scaling scenarios, users should accept delivery partial replicas preferentially.

[RainbowMango]

I'm curious about how these pods be scaled up. By manual, or by something like HPA?

[LivingCcj]

most of scaling up coming from paas platform users.

[LivingCcj]

related code：

karmada/pkg/scheduler/core/division_algorithm.go

Lines 75 to 99 in 8d1041b

	func dynamicDivideReplicas(state *assignState) ([]workv1alpha2.TargetCluster, error) {
	if state.availableReplicas < state.targetReplicas {
	return nil, &framework.UnschedulableError{Message: fmt.Sprintf("Clusters available replicas %d are not enough to schedule.", state.availableReplicas)}
	}
	switch state.strategyType {
	case AggregatedStrategy:
	state.availableClusters = state.resortAvailableClusters()
	var sum int32
	for i := range state.availableClusters {
	if sum += state.availableClusters[i].Replicas; sum >= state.targetReplicas {
	state.availableClusters = state.availableClusters[:i+1]
	break
	}
	}
	fallthrough
	case DynamicWeightStrategy:
	// Set the availableClusters as the weight, scheduledClusters as init result, target as the dispenser object.
	// After dispensing, the target cluster will be the combination of init result and weighted result for target replicas.
	return helper.SpreadReplicasByTargetClusters(state.targetReplicas, state.availableClusters, state.scheduledClusters), nil
	default:
	// should never happen
	return nil, fmt.Errorf("undefined strategy type: %s", state.strategyType)
	}
	}

[RainbowMango]

@kevin-wangzefeng @whitewindmills Please take a look this.

[whitewindmills]

sound reasonable. but if this scenario violates the spread constraint, the user's scheduling request will still be rejected. is this in line with your expectations?

[LivingCcj]

Thanks for your response. the scenario that the pp with spread constraint will always be rejected in the Select scheduler step, due there is no enough available pods. However we have many scenarios that the pp without spread constraint, that expects incremental delivery replicas.

[whitewindmills]

However we have many scenarios that the pp without spread constraint, that expects incremental delivery replicas.

I agree with this. referring to K8s workload scheduling mechanism, pending replicas do not affect schedulable replicas.

[LivingCcj]

The feature could solve effectively the following scenarios
Scenario 1: when the traffic of services is growing, PAAS users or fed hpa controller want to scale up 100 pods to support the growing traffic. however the member clusters only place 95 pods, If the karmada-scheduler delivers the 95 pods, the service would handle the growing traffic, otherwise It is highly likely to crash under excessive overloaded.
Scenario 2：when one member cluster meets crash, some workloads in this cluster need to migrate to other member clusters, which will need many available pods, however the other member cluster can not place all migrated pods. the remaining pods will be unable to handle all the traffic.
@RainbowMango