Support multi-scope configuration settings #10300

abh1sar · 2025-01-29T17:29:05Z

Description

This PR introduces the concept of multi-scope configuration settings. In addition to the Global level, currently all configurations can be set at a single scope level.
It will be useful if a configuration can be set at multiple scopes. For example, a configuration set at the domain level
will apply for all accounts, but it can be set for an account as well. In which case the account level setting will override the domain level setting.

This is done by changing the column scope of table configuration from string (single scope) to bitmask (multiple scopes).

public enum Scope {
    Global(null, 1),
    Zone(Global, 1 << 1),
    Cluster(Zone, 1 << 2),
    StoragePool(Cluster, 1 << 3),
    ManagementServer(Global, 1 << 4),
    ImageStore(Zone, 1 << 5),
    Domain(Global, 1 << 6),
    Account(Domain, 1 << 7);

Each scope is also assigned a parent scope. When a configuration for a given scope is not defined but is available for multiple scope types, the value will be retrieved from the parent scope. If there is no parent scope or if the configuration is defined for a single scope only, the value will fall back to the global level.

Hierarchy for different scopes is defined as below :

Global
- Zone
  - Cluster
    - Storage Pool
  - Image Store
- Management Server
- Domain
  - Account

This PR also updates the scope of the following configurations (Storage Pool scope is added in addition to the existing Zone scope):

pool.storage.allocated.capacity.disablethreshold
pool.storage.allocated.resize.capacity.disablethreshold
pool.storage.capacity.disablethreshold

Doc PR : apache/cloudstack-documentation#476

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Tested Upgrade path
Tested that the configuration values are set correctly for all mentioned configurations at different scopes.

How did you try to break this feature and the system with this change?

Signed-off-by: Abhishek Kumar <[email protected]>

codecov · 2025-01-29T17:36:42Z

Codecov Report

Attention: Patch coverage is 52.92096% with 137 lines in your changes missing coverage. Please review.

Project coverage is 16.07%. Comparing base (cd81398) to head (e20bc6e).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
...ava/com/cloud/upgrade/dao/Upgrade42010to42100.java	0.00%	24 Missing ⚠️
.../apache/cloudstack/framework/config/ConfigKey.java	78.70%	11 Missing and 12 partials ⚠️
...udstack/framework/config/impl/ConfigDepotImpl.java	20.00%	19 Missing and 1 partial ⚠️
...va/com/cloud/upgrade/dao/DatabaseAccessObject.java	65.38%	7 Missing and 2 partials ⚠️
...m/cloud/storage/dao/StoragePoolDetailsDaoImpl.java	0.00%	8 Missing ⚠️
.../main/java/com/cloud/dc/ClusterDetailsDaoImpl.java	0.00%	6 Missing ⚠️
...ain/java/com/cloud/user/AccountDetailsDaoImpl.java	0.00%	6 Missing ⚠️
...storage/datastore/db/ImageStoreDetailsDaoImpl.java	0.00%	6 Missing ⚠️
...ain/java/com/cloud/upgrade/dao/DbUpgradeUtils.java	54.54%	3 Missing and 2 partials ⚠️
.../cloud/configuration/ConfigurationManagerImpl.java	54.54%	2 Missing and 3 partials ⚠️
... and 10 more

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #10300      +/-   ##
============================================
+ Coverage     16.02%   16.07%   +0.05%     
- Complexity    13147    13180      +33     
============================================
  Files          5658     5658              
  Lines        496312   496523     +211     
  Branches      60109    60137      +28     
============================================
+ Hits          79527    79829     +302     
+ Misses       407943   407799     -144     
- Partials       8842     8895      +53

Flag	Coverage Δ
uitests	`4.01% <ø> (ø)`
unittests	`16.92% <52.92%> (+0.05%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

abh1sar · 2025-01-30T16:29:22Z

@blueorangutan package

blueorangutan · 2025-01-30T16:30:03Z

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-01-30T17:37:32Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 12277

abh1sar · 2025-01-30T20:14:20Z

@blueorangutan package

blueorangutan · 2025-01-30T20:16:03Z

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-01-30T21:22:32Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12278

abh1sar · 2025-01-31T05:09:32Z

@blueorangutan package

blueorangutan · 2025-01-31T05:10:03Z

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-01-31T06:13:28Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12282

…udstack into multi-scope

abh1sar · 2025-02-03T05:22:11Z

@blueorangutan package

blueorangutan · 2025-02-03T05:24:04Z

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-02-03T06:26:35Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12301

abh1sar · 2025-02-03T06:53:21Z

@blueorangutan test

blueorangutan · 2025-02-03T06:54:03Z

@abh1sar a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-02-04T00:54:09Z

[SF] Trillian test result (tid-12265)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 62469 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10300-t12265-kvm-ol8.zip
Smoke tests completed. 139 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_11_isolated_network_with_dynamic_routed_mode	`Error`	2.26	test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode	`Error`	3.37	test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode	`Error`	3.37	test_ipv4_routing.py
test_12_start_vm_multiple_volumes_allocated	`Error`	20.07	test_vm_life_cycle.py

shwstppr

Looks goog in general. Some comments added @abh1sar

shwstppr · 2025-02-04T06:24:57Z

engine/schema/src/main/java/com/cloud/upgrade/dao/DbUpgradeUtils.java

+
+    public static void changeTableColumnIfNotExist(Connection conn, String tableName, String oldColumnName, String newColumnName, String columnDefinition) {
+        if (dao.columnExists(conn, tableName, oldColumnName)) {
+            System.out.println("column exists------------------------" + oldColumnName);


this can be removed?

shwstppr · 2025-02-04T06:25:33Z

engine/schema/src/main/java/com/cloud/upgrade/dao/Upgrade42010to42100.java

+
+    protected void migrateConfigurationScopeToBitmask(Connection conn) {
+        String scopeDataType = DbUpgradeUtils.getTableColumnType(conn, "configuration", "scope");
+        logger.info("------------------------{}", scopeDataType);


this can be removed or rephrased if needed

shwstppr · 2025-02-04T06:29:27Z

framework/config/src/main/java/org/apache/cloudstack/framework/config/impl/ConfigurationVO.java

@@ -183,10 +185,15 @@ public void setDescription(String description) {
    }

    @Override
-    public String getScope() {
+    public int getScope() {


@abh1sar we have defined scope as Integer but returning int. Please check if this won't cause an issue

It is not a problem as long as scope is not null which can't happen as the column is declared as not null.

abh1sar · 2025-02-04T08:32:16Z

@blueorangutan package

blueorangutan · 2025-02-04T08:34:04Z

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-02-04T09:51:36Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12321

harikrishna-patnala · 2025-02-05T10:27:25Z

@blueorangutan test

blueorangutan · 2025-02-05T10:28:05Z

@harikrishna-patnala a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-02-06T03:16:13Z

[SF] Trillian test result (tid-12315)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 58031 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10300-t12315-kvm-ol8.zip
Smoke tests completed. 139 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_11_isolated_network_with_dynamic_routed_mode	`Error`	2.34	test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode	`Error`	3.47	test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode	`Error`	3.47	test_ipv4_routing.py
test_12_start_vm_multiple_volumes_allocated	`Error`	13.98	test_vm_life_cycle.py

shwstppr and others added 5 commits January 16, 2025 13:54

wip: multi-scope config

45ad210

Signed-off-by: Abhishek Kumar <[email protected]>

wip: config-key scope as bitmask

c5da8fa

Signed-off-by: Abhishek Kumar <[email protected]>

Minor service layer changes

6e7fe6e

Unit Tests for multi scope config

24d4cfb

revert changes to marvin/setup.py

80f2554

boring-cyborg bot added component:api component:database component:management-server component:networking component:storage component:vpc labels Jan 29, 2025

abh1sar changed the title ~~[WIP] Support multiple hierarchical scopes for configurations~~ [WIP] Add allocation and disable thresholds per storage pool Jan 30, 2025

fixed 41720to41800 upgrade path

3f76576

Merge branch 'main' into multi-scope-config

1924410

abh1sar added 6 commits January 31, 2025 14:30

UT for upgrade path files

16add72

Merge branch 'multi-scope-config' of https://github.com/shapeblue/clo…

3f1c70e

…udstack into multi-scope

fix pre-commit failure

17a9e60

Merge remote-tracking branch 'upstream/main' into multi-scope

d648900

remove unused imports in CapacityManager

82fede2

Add license to ConfigurationGroupsAggregatorTest.java

1666aae

remove extra logging

673dc56

abh1sar changed the title ~~[WIP] Add allocation and disable thresholds per storage pool~~ Add allocation and disable thresholds per storage pool Feb 3, 2025

abh1sar requested review from shwstppr and harikrishna-patnala February 3, 2025 05:07

abh1sar mentioned this pull request Feb 3, 2025

Support multi-scope configuration settings apache/cloudstack-documentation#476

Draft

Move dao code from ConfigurationGroupsAggregator to ConfigurationDaoImpl

e9214ca

shwstppr reviewed Feb 4, 2025

View reviewed changes

abh1sar added 3 commits February 4, 2025 12:12

Fix logging message in Upgrade42010to42100

2fe63ed

Merge remote-tracking branch 'upstream/main' into multi-scope

2d25b7b

fix errors in ConfigurationGroupsAggregatorTest

4d24432

abh1sar requested a review from shwstppr February 4, 2025 06:50

fix errors in MockConfigurationDaoImpl

bab660f

changed logger.debug to logger.warn in DatabaseAccessObject.java

e20bc6e

abh1sar changed the title ~~Add allocation and disable thresholds per storage pool~~ Support multi-scope configuration settings Feb 5, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support multi-scope configuration settings #10300

Support multi-scope configuration settings #10300

abh1sar commented Jan 29, 2025 •

edited

Loading

codecov bot commented Jan 29, 2025 •

edited

Loading

abh1sar commented Jan 30, 2025

blueorangutan commented Jan 30, 2025

blueorangutan commented Jan 30, 2025

abh1sar commented Jan 30, 2025

blueorangutan commented Jan 30, 2025

blueorangutan commented Jan 30, 2025

abh1sar commented Jan 31, 2025

blueorangutan commented Jan 31, 2025

blueorangutan commented Jan 31, 2025

abh1sar commented Feb 3, 2025

blueorangutan commented Feb 3, 2025

blueorangutan commented Feb 3, 2025

abh1sar commented Feb 3, 2025

blueorangutan commented Feb 3, 2025

blueorangutan commented Feb 4, 2025

shwstppr left a comment

shwstppr Feb 4, 2025

shwstppr Feb 4, 2025

shwstppr Feb 4, 2025

abh1sar Feb 4, 2025

abh1sar commented Feb 4, 2025

blueorangutan commented Feb 4, 2025

blueorangutan commented Feb 4, 2025

harikrishna-patnala commented Feb 5, 2025

blueorangutan commented Feb 5, 2025

blueorangutan commented Feb 6, 2025

Support multi-scope configuration settings #10300

Are you sure you want to change the base?

Support multi-scope configuration settings #10300

Conversation

abh1sar commented Jan 29, 2025 • edited Loading

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

codecov bot commented Jan 29, 2025 • edited Loading

Codecov Report

abh1sar commented Jan 30, 2025

blueorangutan commented Jan 30, 2025

blueorangutan commented Jan 30, 2025

abh1sar commented Jan 30, 2025

blueorangutan commented Jan 30, 2025

blueorangutan commented Jan 30, 2025

abh1sar commented Jan 31, 2025

blueorangutan commented Jan 31, 2025

blueorangutan commented Jan 31, 2025

abh1sar commented Feb 3, 2025

blueorangutan commented Feb 3, 2025

blueorangutan commented Feb 3, 2025

abh1sar commented Feb 3, 2025

blueorangutan commented Feb 3, 2025

blueorangutan commented Feb 4, 2025

shwstppr left a comment

Choose a reason for hiding this comment

shwstppr Feb 4, 2025

Choose a reason for hiding this comment

shwstppr Feb 4, 2025

Choose a reason for hiding this comment

shwstppr Feb 4, 2025

Choose a reason for hiding this comment

abh1sar Feb 4, 2025

Choose a reason for hiding this comment

abh1sar commented Feb 4, 2025

blueorangutan commented Feb 4, 2025

blueorangutan commented Feb 4, 2025

harikrishna-patnala commented Feb 5, 2025

blueorangutan commented Feb 5, 2025

blueorangutan commented Feb 6, 2025

abh1sar commented Jan 29, 2025 •

edited

Loading

codecov bot commented Jan 29, 2025 •

edited

Loading