Skip to content

Commit 2deeb83

Browse files
LordHepipudLordHepipud
authored
Merge pull request #142 from Icinga/feature/add_code_cache_for_faster_framework_loading
Experimental: Adds code caching for faster framework loading ## Current Situation Currently the entire Framework is taking a huge amount of time to load the files. This causes several issues: * Slow loading increases the runtime of checks * High CPU usage for the initial loading of the Framework * High CPU usage over a longer period of time, cause more impcact on the systems ## Possible solutions ### Add caching To reduce the impact for the Framework loading, we could add a cache file containing all Cmdlets, Enums and Functions allowing us to import file on initialization instead of having of to search for all `.psm1` files and load them one by one ### Use Nested Modules for PowerShell Plugins PowerShell plugins right now are using `Use-IcingaPlugins` which searchs for all `.psm1` files inside the plugin folder to load them. We should use `NestedModules` here, as the overall impact is lower. On Framework side we can't do this how ever without loading times to explode. Plugins are tracked [at issue #87 here](Icinga/icinga-powershell-plugins#87) ## Current Status **Mitigated** ## Usage You can enable/disable this feature by using `Enable-IcingaFrameworkCodeCache` and `Disable-IcingaFrameworkCodeCache`. Updating the cache is done with `Write-IcingaFrameworkCodeCache`
2 parents fd4d634 + 8d4e66f commit 2deeb83

8 files changed

+153
-10
lines changed

‎doc/31-Changelog.md

+1
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ Released closed milestones can be found on [GitHub](https://github.com/Icinga/ic
1818
* [#139](https://github.com/Icinga/icinga-powershell-framework/pull/139) Add Cmdlet `Start-IcingaShellAsUser` to open an Icinga Shell as different user for testing
1919
* [#141](https://github.com/Icinga/icinga-powershell-framework/pull/141) Adds Cmdlet `Convert-IcingaPluginThresholds` as generic approach to convert Icinga Thresholds with units to the lowest unit of this type.
2020
* [#134](https://github.com/Icinga/icinga-powershell-framework/pull/134) Adds Cmdlet `Test-IcingaWindowsInformation` to check if a WMI class exist and if we can fetch data from it. In addition we add support for binary value comparison with the new Cmdlet `Test-IcingaBinaryOperator`
21+
* [#142](https://github.com/Icinga/icinga-powershell-framework/pull/142) **Experimental:** Adds feature to cache the Framework code into a single file to speed up the entire loading process, mitigating the impact on performance on systems with few CPU cores. You enable disables this feature by using `Enable-IcingaFrameworkCodeCache` and `Disable-IcingaFrameworkCodeCache`. Updating the cache is done with `Write-IcingaFrameworkCodeCache`
2122
* [#149](https://github.com/Icinga/icinga-powershell-framework/pull/149) Adds support to add Wmi permissions for a specific user and namespace with `Add-IcingaWmiPermissions`. In addition you can remove users from Wmi namespaces by using `Remove-IcingaWmiPermissions`
2223

2324
### Bugfixes

‎icinga-powershell-framework.psd1

+20-2
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,21 @@
77
Copyright = '(c) 2020 Icinga GmbH | MIT'
88
Description = 'Icinga for Windows module which allows to entirely monitor the Windows Host system.'
99
PowerShellVersion = '4.0'
10+
NestedModules = @(
11+
'.\lib\core\framework\Get-IcingaFrameworkCodeCache.psm1',
12+
'.\lib\config\Get-IcingaPowerShellConfig.psm1',
13+
'.\lib\config\Read-IcingaPowerShellConfig.psm1',
14+
'.\lib\config\Test-IcingaPowerShellConfigItem.psm1',
15+
'.\lib\core\logging\Write-IcingaConsoleOutput.psm1',
16+
'.\lib\core\logging\Write-IcingaConsoleNotice.psm1',
17+
'.\lib\core\logging\Write-IcingaConsoleWarning.psm1'
18+
)
1019
FunctionsToExport = @(
1120
'Use-Icinga',
1221
'Invoke-IcingaCommand',
1322
'Import-IcingaLib',
23+
'Get-IcingaFrameworkCodeCacheFile',
24+
'Write-IcingaFrameworkCodeCache',
1425
'Publish-IcingaModuleManifest',
1526
'Publish-IcingaEventlogDocumentation',
1627
'Get-IcingaPluginDir',
@@ -19,9 +30,16 @@
1930
'Get-IcingaPowerShellConfigDir',
2031
'Get-IcingaFrameworkRootPath',
2132
'Get-IcingaPowerShellModuleFile',
22-
'Start-IcingaShellAsUser'
33+
'Start-IcingaShellAsUser',
34+
'Get-IcingaPowerShellConfig',
35+
'Get-IcingaFrameworkCodeCache',
36+
'Read-IcingaPowerShellConfig',
37+
'Test-IcingaPowerShellConfigItem',
38+
'Write-IcingaConsoleOutput',
39+
'Write-IcingaConsoleNotice',
40+
'Write-IcingaConsoleWarning'
2341
)
24-
CmdletsToExport = @()
42+
CmdletsToExport = @('*')
2543
VariablesToExport = '*'
2644
AliasesToExport = @( 'icinga' )
2745
PrivateData = @{

‎icinga-powershell-framework.psm1

+59-6
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,10 @@ function Use-Icinga()
2222
Use-IcingaPlugins;
2323
}
2424

25+
if ((Test-Path (Get-IcingaFrameworkCodeCacheFile)) -eq $FALSE -And (Get-IcingaFrameworkCodeCache)) {
26+
Write-IcingaFrameworkCodeCache;
27+
}
28+
2529
# This function will allow us to load this entire module including possible
2630
# actions, making it available within our shell environment
2731
# First load our custom modules
@@ -78,6 +82,20 @@ function Use-Icinga()
7882
}
7983
}
8084

85+
function Get-IcingaFrameworkCodeCacheFile()
86+
{
87+
return (Join-Path -Path (Get-IcingaCacheDir) -ChildPath 'framework_cache.psm1');
88+
}
89+
90+
function Write-IcingaFrameworkCodeCache()
91+
{
92+
if (Get-IcingaFrameworkCodeCache) {
93+
Import-IcingaLib '\' -Init -CompileCache;
94+
} else {
95+
Write-IcingaConsoleNotice 'The experimental code caching feature is currently not enabled. You can enable it with "Enable-IcingaFrameworkCodeCache"';
96+
}
97+
}
98+
8199
function Import-IcingaLib()
82100
{
83101
param(
@@ -87,14 +105,26 @@ function Import-IcingaLib()
87105
[Switch]$ForceReload,
88106
[switch]$Init,
89107
[switch]$Custom,
90-
[switch]$WriteManifests
108+
[switch]$WriteManifests,
109+
[switch]$CompileCache
91110
);
92111

112+
93113
# This is just to only allow a global loading of the module. Import-IcingaLib is ignored on every other
94114
# location. It is just there to give a basic idea within commands, of which functions are used
95115
if ($Init -eq $FALSE) {
96116
return;
97117
}
118+
119+
$CacheFile = Get-IcingaFrameworkCodeCacheFile;
120+
121+
if ($Custom -eq $FALSE -And $CompileCache -eq $FALSE -And (Test-Path $CacheFile) -And (Get-IcingaFrameworkCodeCache)) {
122+
Import-Module $CacheFile -Global;
123+
return;
124+
}
125+
126+
[array]$ImportModules = @();
127+
[array]$RemoveModules = @();
98128

99129
if ($Custom) {
100130
[string]$directory = Join-Path -Path $PSScriptRoot -ChildPath 'custom\';
@@ -116,11 +146,11 @@ function Import-IcingaLib()
116146

117147
if ($ListOfLoadedModules -like "*$moduleName*") {
118148
if ($ForceReload) {
119-
Remove-Module -Name $moduleName
120-
Import-Module ([string]::Format('{0}', $modulePath)) -Global;
149+
$RemoveModules += $moduleName;
121150
}
151+
$ImportModules += $modulePath;
122152
} else {
123-
Import-Module ([string]::Format('{0}', $modulePath)) -Global;
153+
$ImportModules += $modulePath;
124154
if ($WriteManifests) {
125155
Publish-IcingaModuleManifest -Module $moduleName;
126156
}
@@ -132,15 +162,35 @@ function Import-IcingaLib()
132162

133163
if ($ForceReload) {
134164
if ($ListOfLoadedModules -Like "*$moduleName*") {
135-
Remove-Module -Name $moduleName;
165+
$RemoveModules += $moduleName;
136166
}
137167
}
138168

139-
Import-Module ([string]::Format('{0}.psm1', $module)) -Global;
169+
$ImportModules += ([string]::Format('{0}.psm1', $module));
140170
if ($WriteManifests) {
141171
Publish-IcingaModuleManifest -Module $moduleName;
142172
}
143173
}
174+
175+
if ($RemoveModules.Count -ne 0) {
176+
Remove-Module $RemoveModules;
177+
}
178+
179+
if ($ImportModules.Count -ne 0) {
180+
181+
if ($CompileCache) {
182+
$CacheContent = '';
183+
foreach ($module in $ImportModules) {
184+
$Content = Get-Content $module -Raw;
185+
$CacheContent += $Content + "`r`n";
186+
}
187+
188+
$CacheContent += $Content + "Export-ModuleMember -Function @( '*' )";
189+
Set-Content -Path $CacheFile -Value $CacheContent;
190+
} else {
191+
Import-Module $ImportModules -Global;
192+
}
193+
}
144194
}
145195

146196
function Publish-IcingaModuleManifest()
@@ -279,6 +329,9 @@ function Invoke-IcingaCommand()
279329
Write-Output ([string]::Format('** Icinga PowerShell Framework {0}', $IcingaFrameworkData.PrivateData.Version));
280330
Write-Output ([string]::Format('** Copyright {0}', $IcingaFrameworkData.Copyright));
281331
Write-Output ([string]::Format('** User environment {0}\{1}', $env:USERDOMAIN, $env:USERNAME));
332+
if (Get-IcingaFrameworkCodeCache) {
333+
Write-Output ([string]::Format('** Warning: Icinga Framework Code Caching is enabled'));
334+
}
282335
Write-Output '******************************************************';
283336
}
284337

‎lib/config/Read-IcingaPowerShellConfig.psm1

+2-2
Original file line numberDiff line numberDiff line change
@@ -25,13 +25,13 @@ function Read-IcingaPowerShellConfig()
2525
}
2626

2727
if (-Not (Test-Path $ConfigFile)) {
28-
return (New-Object -TypeName PSOBject);
28+
return (New-Object -TypeName PSObject);
2929
}
3030

3131
[string]$Content = Get-Content -Path $ConfigFile;
3232

3333
if ([string]::IsNullOrEmpty($Content)) {
34-
return (New-Object -TypeName PSOBject);
34+
return (New-Object -TypeName PSObject);
3535
}
3636

3737
return (ConvertFrom-Json -InputObject $Content);

‎lib/core/framework/Disable-IcingaFrameworkCodeCache.psm1

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
<#
2+
.SYNOPSIS
3+
Disables the experimental feature to cache all functions into a single file,
4+
allowing quicker loading times of the Icinga PowerShell Framework
5+
.DESCRIPTION
6+
Disables the experimental feature to cache all functions into a single file,
7+
allowing quicker loading times of the Icinga PowerShell Framework
8+
.FUNCTIONALITY
9+
Experimental: Disables the Icinga for Windows code caching
10+
.EXAMPLE
11+
PS>Disable-IcingaFrameworkCodeCache;
12+
.LINK
13+
https://github.com/Icinga/icinga-powershell-framework
14+
#>
15+
16+
function Disable-IcingaFrameworkCodeCache()
17+
{
18+
Set-IcingaPowerShellConfig -Path 'Framework.Experimental.CodeCaching' -Value $FALSE;
19+
}

‎lib/core/framework/Enable-IcingaFrameworkCodeCache.psm1

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
<#
2+
.SYNOPSIS
3+
Enables the experimental feature to cache all functions into a single file,
4+
allowing quicker loading times of the Icinga PowerShell Framework
5+
.DESCRIPTION
6+
Enables the experimental feature to cache all functions into a single file,
7+
allowing quicker loading times of the Icinga PowerShell Framework
8+
.FUNCTIONALITY
9+
Experimental: Enables the Icinga for Windows code caching
10+
.EXAMPLE
11+
PS>Enable-IcingaFrameworkCodeCache;
12+
.LINK
13+
https://github.com/Icinga/icinga-powershell-framework
14+
#>
15+
16+
function Enable-IcingaFrameworkCodeCache()
17+
{
18+
Set-IcingaPowerShellConfig -Path 'Framework.Experimental.CodeCaching' -Value $TRUE;
19+
20+
Write-IcingaConsoleWarning 'This is an experimental feature and might cause some side effects during usage. Please use this function with caution. Please run "Write-IcingaFrameworkCodeCache" in addition to ensure your cache is updated. This should be done after each update of the Framework in case the feature was disabled during the update run.';
21+
}

‎lib/core/framework/Get-IcingaFrameworkCodeCache.psm1

+28
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
<#
2+
.SYNOPSIS
3+
Fetches the current enable/disable state of the experimental feature
4+
for caching the Icinga PowerShell Framework code
5+
.DESCRIPTION
6+
Fetches the current enable/disable state of the experimental feature
7+
for caching the Icinga PowerShell Framework code
8+
.FUNCTIONALITY
9+
Experimental: Get the current code caching configuration of the
10+
Icinga PowerShell Framework
11+
.EXAMPLE
12+
PS>Get-IcingaFrameworkCodeCache;
13+
.LINK
14+
https://github.com/Icinga/icinga-powershell-framework
15+
.OUTPUTS
16+
System.Boolean
17+
#>
18+
19+
function Get-IcingaFrameworkCodeCache()
20+
{
21+
$CodeCaching = Get-IcingaPowerShellConfig -Path 'Framework.Experimental.CodeCaching';
22+
23+
if ($null -eq $CodeCaching) {
24+
return $FALSE;
25+
}
26+
27+
return $CodeCaching;
28+
}

‎lib/core/framework/Install-IcingaFrameworkUpdate.psm1

+3
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,9 @@ function Install-IcingaFrameworkUpdate()
9191
Start-Sleep -Seconds 1;
9292
Remove-ItemSecure -Path $Archive.Directory -Recurse -Force | Out-Null;
9393

94+
Write-IcingaConsoleNotice 'Updating Framework cache file';
95+
Write-IcingaFrameworkCodeCache;
96+
9497
Write-IcingaConsoleNotice 'Framework update has been completed. Please start a new PowerShell instance now to complete the update';
9598

9699
Test-IcingaAgent;

0 commit comments

Comments
 (0)