APCng storage - removed APCUIterator usage while collecting metrics, removed metrics collecting lag (#96)

* APCng storage - removed APCUIterator usage while collecting metrics, removed metrics collecting lag

Signed-off-by: Rastusik <[email protected]>

* fixed tests for APCng storage changes with no need for TTL for meta info generation

Signed-off-by: Rastusik <[email protected]>

* allowed phpstan extension installer plugin in composer, so CI pipeline can work again

Signed-off-by: Rastusik <[email protected]>

* APCng - some little optimisations for persistent memory workloads (e.g. swoole), better concurrent data updates, fixed tests so they work with floats correctly

Signed-off-by: Rastusik <[email protected]>

* small fixes for phpstan runner to pass

Signed-off-by: Rastusik <[email protected]>

* acpng store fix - all data have fixed ttl of 0, so no global apcu setting can make the metrics expire

Signed-off-by: Rastusik <[email protected]>

Signed-off-by: Rastusik <[email protected]>

Author: Martin Fris

.github/workflows/tests.yml

@@ -35,7 +35,7 @@ jobs:
         with:
           php-version: ${{ matrix.php }}
           extensions: redis, apcu
-          ini-values: apc.enable_cli='On'
+          ini-values: apc.enable_cli='On',apc.shm_size = 256M
           coverage: none
 
       - name: Install dependencies

README.APCng.md

@@ -62,5 +62,3 @@ Without going into excruciating detail (you can read the source for that!), the
 The approach `APCng` takes is to keep a "metadata cache" which stores an array of all the metadata keys, so instead of doing a scan of APCu looking for all matching keys, we just need to retrieve one key, deserialize it (which turns out to be slow), and retrieve all the metadata keys listed in the array.  Once we've done that, there is some fancy handwaving which is used to deterministically generate possible sub-keys for each metadata item, based on LabelNames, etc. Not all of these keys exist, but it's quicker to attempt to fetch them and fail, then it is to run another APCUIterator looking for a specific pattern.
 
 Summaries, as mentioned before, have a third nested APCUIterator in them, looking for all readings w/o expired TTLs that match a pattern.  Again, slow.  Instead, we store a "map", similar to the metadata cache, but this one is temporally-keyed: one key per second, which lists how many samples were collected in that second. Once this is done, an expensive APCUIterator match is no longer needed, as all possible keys can be deterministically generated and checked, by retrieving each key for the past 600 seconds (if it exists), extracting the sample-count from the key, and then generating all the APCu keys which would refer to each observed sample.
-
-There is the concept of a metadata cache TTL (default: 1 second) which offers a trade-off of performance vs responsiveness. If a collect() call is made and then a new metric is subsequently tracked, the new metric won't show up in subsequent collect() calls until the metadata cache TTL is expired. By keeping this TTL short, we avoid hammering APCu too heavily (remember, deserializing that metainfo cache array is nearly as slow as calling APCUIterator -- it just doesn't slow down as you add more keys to APCu). However we want to cap how long a new metric remains "hidden" from the Prometheus scraper.  For best performance, adjust the TTL as high as you can based on your specific use-case. For instance if you're scraping every 10 seconds, then a reasonable TTL could be anywhere from 5 to 10 seconds, meaning a 50 to 100% chance that the metric won't appear in the next full scrape, but it will be there for the following one.  Note that the data is tracked just fine during this period - it's just not visible yet, but it will be!  You can set the TTL to zero to disable the cache. This will return to `APC` engine behavior, with no delay between creating a metric and being able to collect() it. However, performance will suffer, as the metainfo cache array will need to be deserialized from APCu each time collect() is called -- which might be okay if collect() is called infrequently and you simply must have zero delay in reporting newly-created metrics.

composer.json

@@ -50,7 +50,10 @@
         }
     },
     "config": {
-        "sort-packages": true
+        "sort-packages": true,
+        "allow-plugins": {
+            "phpstan/extension-installer": true
+        }
     },
     "prefer-stable": true
 }

php-fpm/docker-php-ext-apcu-cli.ini

@@ -1 +1,2 @@
 apc.enable_cli = On
+apc.shm_size = 256M

src/Prometheus/RenderTextFormat.php

@@ -4,6 +4,8 @@
 
 namespace Prometheus;
 
+use RuntimeException;
+
 class RenderTextFormat implements RendererInterface
 {
     const MIME_TYPE = 'text/plain; version=0.0.4';
@@ -65,6 +67,11 @@ private function escapeAllLabels(array $labelNames, Sample $sample): array
 
         $labels = array_combine(array_merge($labelNames, $sample->getLabelNames()), $sample->getLabelValues());
 
+        /** @phpstan-ignore-next-line */
+        if ($labels === false) {
+            throw new RuntimeException('Unable to combine labels.');
+        }
+
         foreach ($labels as $labelName => $labelValue) {
             $escapedLabels[] = $labelName . '="' . $this->escapeLabelValue((string)$labelValue) . '"';
         }

src/Prometheus/Storage/APC.php

@@ -4,7 +4,7 @@
 
 namespace Prometheus\Storage;
 
-use APCUIterator;
+use APCuIterator;
 use Prometheus\Exception\StorageException;
 use Prometheus\Math;
 use Prometheus\MetricFamilySamples;
@@ -178,7 +178,7 @@ public function wipeStorage(): void
         //                        .+  | at least one additional character
         $matchAll = sprintf('/^%s:.+/', $this->prometheusPrefix);
 
-        foreach (new APCUIterator($matchAll) as $key => $value) {
+        foreach (new APCuIterator($matchAll) as $key => $value) {
             apcu_delete($key);
         }
     }
@@ -241,7 +241,7 @@ private function metaData(array $data): array
     private function collectCounters(): array
     {
         $counters = [];
-        foreach (new APCUIterator('/^' . $this->prometheusPrefix . ':counter:.*:meta/') as $counter) {
+        foreach (new APCuIterator('/^' . $this->prometheusPrefix . ':counter:.*:meta/') as $counter) {
             $metaData = json_decode($counter['value'], true);
             $data = [
                 'name' => $metaData['name'],
@@ -250,7 +250,7 @@ private function collectCounters(): array
                 'labelNames' => $metaData['labelNames'],
                 'samples' => [],
             ];
-            foreach (new APCUIterator('/^' . $this->prometheusPrefix . ':counter:' . $metaData['name'] . ':.*:value/') as $value) {
+            foreach (new APCuIterator('/^' . $this->prometheusPrefix . ':counter:' . $metaData['name'] . ':.*:value/') as $value) {
                 $parts = explode(':', $value['key']);
                 $labelValues = $parts[3];
                 $data['samples'][] = [
@@ -272,7 +272,7 @@ private function collectCounters(): array
     private function collectGauges(): array
     {
         $gauges = [];
-        foreach (new APCUIterator('/^' . $this->prometheusPrefix . ':gauge:.*:meta/') as $gauge) {
+        foreach (new APCuIterator('/^' . $this->prometheusPrefix . ':gauge:.*:meta/') as $gauge) {
             $metaData = json_decode($gauge['value'], true);
             $data = [
                 'name' => $metaData['name'],
@@ -281,7 +281,7 @@ private function collectGauges(): array
                 'labelNames' => $metaData['labelNames'],
                 'samples' => [],
             ];
-            foreach (new APCUIterator('/^' . $this->prometheusPrefix . ':gauge:' . $metaData['name'] . ':.*:value/') as $value) {
+            foreach (new APCuIterator('/^' . $this->prometheusPrefix . ':gauge:' . $metaData['name'] . ':.*:value/') as $value) {
                 $parts = explode(':', $value['key']);
                 $labelValues = $parts[3];
                 $data['samples'][] = [
@@ -304,7 +304,7 @@ private function collectGauges(): array
     private function collectHistograms(): array
     {
         $histograms = [];
-        foreach (new APCUIterator('/^' . $this->prometheusPrefix . ':histogram:.*:meta/') as $histogram) {
+        foreach (new APCuIterator('/^' . $this->prometheusPrefix . ':histogram:.*:meta/') as $histogram) {
             $metaData = json_decode($histogram['value'], true);
             $data = [
                 'name' => $metaData['name'],
@@ -318,7 +318,7 @@ private function collectHistograms(): array
             $data['buckets'][] = '+Inf';
 
             $histogramBuckets = [];
-            foreach (new APCUIterator('/^' . $this->prometheusPrefix . ':histogram:' . $metaData['name'] . ':.*:value/') as $value) {
+            foreach (new APCuIterator('/^' . $this->prometheusPrefix . ':histogram:' . $metaData['name'] . ':.*:value/') as $value) {
                 $parts = explode(':', $value['key']);
                 $labelValues = $parts[3];
                 $bucket = $parts[4];
@@ -380,7 +380,7 @@ private function collectSummaries(): array
     {
         $math = new Math();
         $summaries = [];
-        foreach (new APCUIterator('/^' . $this->prometheusPrefix . ':summary:.*:meta/') as $summary) {
+        foreach (new APCuIterator('/^' . $this->prometheusPrefix . ':summary:.*:meta/') as $summary) {
             $metaData = $summary['value'];
             $data = [
                 'name' => $metaData['name'],
@@ -392,11 +392,11 @@ private function collectSummaries(): array
                 'samples' => [],
             ];
 
-            foreach (new APCUIterator('/^' . $this->prometheusPrefix . ':summary:' . $metaData['name'] . ':.*:value$/') as $value) {
+            foreach (new APCuIterator('/^' . $this->prometheusPrefix . ':summary:' . $metaData['name'] . ':.*:value$/') as $value) {
                 $encodedLabelValues = $value['value'];
                 $decodedLabelValues = $this->decodeLabelValues($encodedLabelValues);
                 $samples = [];
-                foreach (new APCUIterator('/^' . $this->prometheusPrefix . ':summary:' . $metaData['name'] . ':' . str_replace('/', '\\/', preg_quote($encodedLabelValues)) . ':value:.*/') as $sample) {
+                foreach (new APCuIterator('/^' . $this->prometheusPrefix . ':summary:' . $metaData['name'] . ':' . str_replace('/', '\\/', preg_quote($encodedLabelValues)) . ':value:.*/') as $sample) {
                     $samples[] = $sample['value'];
                 }