Skip to content

Support multi-scope configuration settings #10300

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 27 commits into from
Feb 14, 2025

Conversation

abh1sar
Copy link
Collaborator

@abh1sar abh1sar commented Jan 29, 2025

Description

This PR introduces the concept of multi-scope configuration settings. In addition to the Global level, currently all configurations can be set at a single scope level.
It will be useful if a configuration can be set at multiple scopes. For example, a configuration set at the domain level
will apply for all accounts, but it can be set for an account as well. In which case the account level setting will override the domain level setting.

This is done by changing the column scope of table configuration from string (single scope) to bitmask (multiple scopes).

public enum Scope {
    Global(null, 1),
    Zone(Global, 1 << 1),
    Cluster(Zone, 1 << 2),
    StoragePool(Cluster, 1 << 3),
    ManagementServer(Global, 1 << 4),
    ImageStore(Zone, 1 << 5),
    Domain(Global, 1 << 6),
    Account(Domain, 1 << 7);

Each scope is also assigned a parent scope. When a configuration for a given scope is not defined but is available for multiple scope types, the value will be retrieved from the parent scope. If there is no parent scope or if the configuration is defined for a single scope only, the value will fall back to the global level.

Hierarchy for different scopes is defined as below :

  • Global
    • Zone
      • Cluster
        • Storage Pool
      • Image Store
    • Management Server
    • Domain
      • Account

This PR also updates the scope of the following configurations (Storage Pool scope is added in addition to the existing Zone scope):

  • pool.storage.allocated.capacity.disablethreshold
  • pool.storage.allocated.resize.capacity.disablethreshold
  • pool.storage.capacity.disablethreshold

Doc PR : apache/cloudstack-documentation#476

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • build/CI
  • test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

Tested Upgrade path
Tested that the configuration values are set correctly for all mentioned configurations at different scopes.

How did you try to break this feature and the system with this change?

Copy link

codecov bot commented Jan 29, 2025

Codecov Report

Attention: Patch coverage is 64.28571% with 105 lines in your changes missing coverage. Please review.

Project coverage is 16.13%. Comparing base (f8563b8) to head (9f22056).
Report is 278 commits behind head on main.

Files with missing lines Patch % Lines
.../apache/cloudstack/framework/config/ConfigKey.java 78.70% 11 Missing and 12 partials ⚠️
...va/com/cloud/upgrade/dao/DatabaseAccessObject.java 65.38% 7 Missing and 2 partials ⚠️
...udstack/framework/config/impl/ConfigDepotImpl.java 64.00% 4 Missing and 5 partials ⚠️
...m/cloud/storage/dao/StoragePoolDetailsDaoImpl.java 0.00% 8 Missing ⚠️
...storage/datastore/db/ImageStoreDetailsDaoImpl.java 0.00% 7 Missing ⚠️
.../main/java/com/cloud/dc/ClusterDetailsDaoImpl.java 0.00% 6 Missing ⚠️
...ain/java/com/cloud/user/AccountDetailsDaoImpl.java 0.00% 6 Missing ⚠️
...ain/java/com/cloud/upgrade/dao/DbUpgradeUtils.java 54.54% 3 Missing and 2 partials ⚠️
...ava/com/cloud/upgrade/dao/Upgrade42010to42100.java 79.16% 4 Missing and 1 partial ⚠️
.../cloud/configuration/ConfigurationManagerImpl.java 54.54% 2 Missing and 3 partials ⚠️
... and 9 more
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #10300      +/-   ##
============================================
+ Coverage     16.07%   16.13%   +0.06%     
- Complexity    13189    13233      +44     
============================================
  Files          5664     5664              
  Lines        497612   497821     +209     
  Branches      60216    60243      +27     
============================================
+ Hits          79991    80335     +344     
+ Misses       408663   408479     -184     
- Partials       8958     9007      +49     
Flag Coverage Δ
uitests 4.00% <ø> (ø)
unittests 16.98% <64.28%> (+0.06%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@abh1sar abh1sar changed the title [WIP] Support multiple hierarchical scopes for configurations [WIP] Add allocation and disable thresholds per storage pool Jan 30, 2025
@abh1sar
Copy link
Collaborator Author

abh1sar commented Jan 30, 2025

@blueorangutan package

@blueorangutan
Copy link

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✖️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 12277

@abh1sar
Copy link
Collaborator Author

abh1sar commented Jan 30, 2025

@blueorangutan package

@blueorangutan
Copy link

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12278

@abh1sar
Copy link
Collaborator Author

abh1sar commented Jan 31, 2025

@blueorangutan package

@blueorangutan
Copy link

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12282

@abh1sar
Copy link
Collaborator Author

abh1sar commented Feb 12, 2025

@blueorangutan package

@blueorangutan
Copy link

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 12428

@abh1sar
Copy link
Collaborator Author

abh1sar commented Feb 13, 2025

@blueorangutan package

@blueorangutan
Copy link

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12430

@abh1sar
Copy link
Collaborator Author

abh1sar commented Feb 13, 2025

@blueorangutan test

@blueorangutan
Copy link

@abh1sar a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

Copy link
Contributor

@borisstoyanov borisstoyanov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, manually tested that pool settings are taking precedence over the global settings.

@rohityadavcloud rohityadavcloud marked this pull request as ready for review February 13, 2025 12:18
@rohityadavcloud
Copy link
Member

@blueorangutan package

@blueorangutan
Copy link

@rohityadavcloud a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12443

@blueorangutan
Copy link

[SF] Trillian test result (tid-12388)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 54507 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10300-t12388-kvm-ol8.zip
Smoke tests completed. 139 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_11_isolated_network_with_dynamic_routed_mode Error 2.35 test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode Error 1.39 test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode Error 1.39 test_ipv4_routing.py
test_06_purge_expunged_vm_background_task Failure 399.20 test_purge_expunged_vms.py

@rohityadavcloud rohityadavcloud merged commit 2a4a1f7 into apache:main Feb 14, 2025
26 checks passed
@rohityadavcloud rohityadavcloud deleted the multi-scope-config branch February 14, 2025 05:55
weizhouapache pushed a commit to shapeblue/cloudstack that referenced this pull request Mar 4, 2025
…che#526)

Introduces multiple scope configuration parameters by changing the column `scope` of table `configuration` from string to bitmask.

Following configuration parameters can now be configured at a Storage Pool scope in addition to the Zone scope.

1. **pool.storage.capacity.disablethreshold**: Percentage (as a value between 0 and 1) of storage utilization or used space relative to the total storage capacity above which the storage pool is disabled for new allocations. migration and resize operations.

2. **pool.storage.allocated.capacity.disablethreshold**: Percentage (as a value between 0 and 1) of allocated storage space relative to total overprovisioned storage capacity above which the storage pool is disabled for new allocations, migration and resize operations.

3. **pool.storage.allocated.resize.capacity.disablethreshold**: Percentage (as a value between 0 and 1) of allocated storage space relative to total overprovisioned storage capacity above which the storage pool is disabled for volume resize. This is applicable only when volume.resize.allowed.beyond.allocation is set to true

4. **volume.resize.allowed.beyond.allocation**: Specifies whether a volume can be resized beyond the pool capacity allocation disable threshold (pool.storage.allocated.capacity.disablethreshold) but not exceeding the resize capacity disable threshold (pool.storage.allocated.resize.capacity.disablethreshold). Please note that resize due to change of disk offering still honours pool capacity allocation disable threshold (pool.storage.allocated.capacity.disablethreshold).
If set, the granular scope (storage pool) overrides the value set for the broader scope (zone). 

If set, the granular scope (storage pool) overrides the value set for the broader scope (zone). 

The operator will see these configurations in the Settings tab under zone as well as under primary storage. listConfigurations storageid=<storage_id> | zoneid=<zone_id> and updateConfigurations storage_id=<storage_id> | zoneid=<zone_id> will also work.

Upstream PR: apache#10300

---------

Co-authored-by: Harikrishna Patnala <[email protected]>
dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Jun 19, 2025
This PR introduces the concept of multi-scope configuration settings. In addition to the Global level, currently all configurations can be set at a single scope level.
It will be useful if a configuration can be set at multiple scopes. For example, a configuration set at the domain level
will apply for all accounts, but it can be set for an account as well. In which case the account level setting will override the domain level setting.

This is done by changing the column `scope` of table `configuration` from string (single scope) to bitmask (multiple scopes).

```
public enum Scope {
    Global(null, 1),
    Zone(Global, 1 << 1),
    Cluster(Zone, 1 << 2),
    StoragePool(Cluster, 1 << 3),
    ManagementServer(Global, 1 << 4),
    ImageStore(Zone, 1 << 5),
    Domain(Global, 1 << 6),
    Account(Domain, 1 << 7);
```
Each scope is also assigned a parent scope. When a configuration for a given scope is not defined but is available for multiple scope types, the value will be retrieved from the parent scope. If there is no parent scope or if the configuration is defined for a single scope only, the value will fall back to the global level.

Hierarchy for different scopes is defined as below :
- Global
    - Zone
        - Cluster
            - Storage Pool
        - Image Store
    - Management Server
    - Domain
        - Account

This PR also updates the scope of the following configurations (Storage Pool scope is added in addition to the existing Zone scope):
- pool.storage.allocated.capacity.disablethreshold
- pool.storage.allocated.resize.capacity.disablethreshold
- pool.storage.capacity.disablethreshold

Doc PR : apache/cloudstack-documentation#476

Signed-off-by: Abhishek Kumar <[email protected]>
Co-authored-by: Abhishek Kumar <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

8 participants