Skip to content

Limit what we iterate over in providers#1704

Open
jstack-godaddy wants to merge 2 commits into
puppetlabs:mainfrom
jstack-godaddy:efficient_iterations
Open

Limit what we iterate over in providers#1704
jstack-godaddy wants to merge 2 commits into
puppetlabs:mainfrom
jstack-godaddy:efficient_iterations

Conversation

@jstack-godaddy
Copy link
Copy Markdown

@jstack-godaddy jstack-godaddy commented Mar 6, 2026

Summary

The current implementation of applying/creating users, grants and databases was creating extremely long puppet runs in one of our environments. The environment is a shared hosting implementation that has 1000+ schema and 4 users per schema for RW/RO access. These schema and users are managed by another team, so we do not want to prune. The other team also does not want to limit users as it would create a split in workflow between on-prem and cloud provisioning. Puppet runs with --tag mysql were taking close to 14 minutes to loop over all users, all grants, and all databases. In this change we are determining what the users, grants and databases we care about and only iterating over them. There's little to gain by looping over everything if we only care about the users, grants, and databases we are managing via hiera and manifests. The result is astounding. The runtime dropped from 14 minutes to 11 seconds with no change in expected behavior.

[root@HOST ~]# puppet agent --test --tags mysql
....
Notice: Applied catalog in 820.98 seconds

after:

[root@HOST ~]# puppet agent --test --tags mysql
....
Notice: Applied catalog in 11.32 seconds

Additional Context

Add any additional context about the problem here.

  • To see long puppet application times, create a large amount of users and schema. In our case 1000+ schema and 4000+ users.
  • Speed up application times, resulting in faster delivery of changes in the manifests and less resource usage on database servers.

Related Issues (if any)

N/A

Checklist

  • 🟢 Spec tests.
  • 🟢 Acceptance tests.
  • Manually verified. (For example puppet apply)

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 6, 2026

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants