benchmarks: Use 1 Warmup Interation and ensure opcache is used

[PROFeNoM]

Description

Native profiles when:

• Running benchSymfonyOverhead (i.e., with tracer) and opcache. We can indeed see some opcache_compile_file calls
• Running benchSymfonyOverhead (i.e., with tracer) without opcache. We cannot see any opcache_compile_file calls

These profiles assess that opcache is indeed used in web requests. What's more, the warmups seemingly responsible for improvements in both kinds of benchmarks, because the first run will be longer since DD_AUTOLOAD_NO_COMPILE is used.

Reviewer checklist

• Test coverage seems ok.
• Appropriate labels assigned.

[pr-commenter]

Benchmarks

Benchmark execution time: 2024-02-23 11:22:48

Comparing candidate commit 4257a5e in PR branch alex/experiments/web-bench-warmups with baseline commit ab2dd7d in branch master.

Found 35 performance improvements and 2 performance regressions! Performance is the same for 144 metrics, 1 unstable metrics.

scenario:ContextPropagationBench/benchExtractHeaders128Bit

• 🟩 execution_time [-56.335ns; -28.998ns] or [-6.273%; -3.229%]

scenario:ContextPropagationBench/benchExtractHeaders128Bit-opcache

• 🟩 execution_time [-84.318ns; -41.015ns] or [-6.008%; -2.923%]

scenario:ContextPropagationBench/benchExtractHeaders64Bit

• 🟩 execution_time [-130.436ns; -91.564ns] or [-9.927%; -6.968%]

scenario:ContextPropagationBench/benchExtractHeaders64Bit-opcache

• 🟩 execution_time [-157.791ns; -112.209ns] or [-11.846%; -8.424%]

scenario:ContextPropagationBench/benchExtractTraceContext128Bit

• 🟩 execution_time [-136.200ns; -93.800ns] or [-5.950%; -4.098%]

scenario:ContextPropagationBench/benchExtractTraceContext128Bit-opcache

• 🟩 execution_time [-168.737ns; -77.263ns] or [-7.311%; -3.348%]

scenario:ContextPropagationBench/benchExtractTraceContext64Bit

• 🟩 execution_time [-70.410ns; -47.590ns] or [-7.992%; -5.402%]

scenario:ContextPropagationBench/benchExtractTraceContext64Bit-opcache

• 🟩 execution_time [-67.493ns; -50.507ns] or [-7.670%; -5.739%]

scenario:ContextPropagationBench/benchInject128Bit-opcache

• 🟩 execution_time [-188.861ns; -117.139ns] or [-6.735%; -4.178%]

scenario:EmptyFileBench/benchEmptyFileBaseline

• 🟩 execution_time [-1.524ms; -1.320ms] or [-40.479%; -35.048%]

scenario:EmptyFileBench/benchEmptyFileBaseline-opcache

• 🟩 execution_time [-1.593ms; -1.395ms] or [-40.197%; -35.212%]

scenario:EmptyFileBench/benchEmptyFileOverhead

• 🟩 execution_time [-6.341ms; -6.108ms] or [-63.166%; -60.850%]

scenario:EmptyFileBench/benchEmptyFileOverhead-opcache

• 🟩 execution_time [-6.257ms; -6.058ms] or [-61.968%; -59.998%]

scenario:LaravelBench/benchLaravelBaseline

• 🟩 execution_time [-1.629ms; -1.395ms] or [-40.607%; -34.780%]

scenario:LaravelBench/benchLaravelBaseline-opcache

• 🟩 execution_time [-1.634ms; -1.419ms] or [-39.245%; -34.084%]

scenario:LaravelBench/benchLaravelOverhead

• 🟩 execution_time [-6.536ms; -6.285ms] or [-62.872%; -60.458%]

scenario:LaravelBench/benchLaravelOverhead-opcache

• 🟩 execution_time [-6.504ms; -6.315ms] or [-62.104%; -60.292%]

scenario:LogsInjectionBench/benchLogsInfoBaseline

• 🟩 execution_time [-140.569ns; -108.431ns] or [-9.954%; -7.678%]

scenario:LogsInjectionBench/benchLogsInfoBaseline-opcache

• 🟩 execution_time [-214.875ns; -144.725ns] or [-11.675%; -7.863%]

scenario:LogsInjectionBench/benchLogsInfoInjection-opcache

• 🟩 execution_time [-517.093ns; -201.507ns] or [-5.137%; -2.002%]

scenario:LogsInjectionBench/benchLogsNullBaseline-opcache

• 🟩 execution_time [-1.199µs; -1.091µs] or [-5.071%; -4.613%]

scenario:LogsInjectionBench/benchLogsNullInjection

• 🟩 execution_time [-2.035µs; -1.253µs] or [-3.811%; -2.346%]

scenario:LogsInjectionBench/benchLogsNullInjection-opcache

• 🟩 execution_time [-2.389µs; -2.295µs] or [-9.604%; -9.224%]

scenario:MessagePackSerializationBench/benchMessagePackSerialization

• 🟩 execution_time [-7.699µs; -5.461µs] or [-4.962%; -3.519%]

scenario:MessagePackSerializationBench/benchMessagePackSerialization-opcache

• 🟩 execution_time [-7.822µs; -6.458µs] or [-5.014%; -4.140%]

scenario:SamplingRuleMatchingBench/benchRegexMatching3-opcache

• 🟥 execution_time [+1.024µs; +1.276µs] or [+16.684%; +20.792%]

scenario:SamplingRuleMatchingBench/benchRegexMatching4-opcache

• 🟥 execution_time [+0.988µs; +1.241µs] or [+16.184%; +20.321%]

scenario:SymfonyBench/benchSymfonyBaseline

• 🟩 execution_time [-14.518ms; -14.278ms] or [-58.987%; -58.009%]

scenario:SymfonyBench/benchSymfonyBaseline-opcache

• 🟩 execution_time [-14.450ms; -14.285ms] or [-58.601%; -57.932%]

scenario:SymfonyBench/benchSymfonyOverhead

• 🟩 execution_time [-19.177ms; -18.964ms] or [-60.970%; -60.295%]

scenario:SymfonyBench/benchSymfonyOverhead-opcache

• 🟩 execution_time [-19.159ms; -19.014ms] or [-60.762%; -60.303%]

scenario:WordPressBench/benchEnhancedWordPressOverhead

• 🟩 execution_time [-55.715ms; -53.784ms] or [-21.179%; -20.445%]

scenario:WordPressBench/benchEnhancedWordPressOverhead-opcache

• 🟩 execution_time [-54.781ms; -53.148ms] or [-20.856%; -20.234%]

scenario:WordPressBench/benchWordPressBaseline

• 🟩 execution_time [-42.885ms; -42.317ms] or [-66.072%; -65.197%]

scenario:WordPressBench/benchWordPressBaseline-opcache

• 🟩 execution_time [-42.479ms; -41.995ms] or [-65.680%; -64.931%]

scenario:WordPressBench/benchWordPressOverhead

• 🟩 execution_time [-56.222ms; -53.934ms] or [-21.406%; -20.535%]

scenario:WordPressBench/benchWordPressOverhead-opcache

• 🟩 execution_time [-56.863ms; -54.138ms] or [-21.619%; -20.583%]

[bwoebi]

I see 20%-60% perf improvement, I approve!

[codecov-commenter]

Codecov Report

Merging #2531 (4257a5e) into master (ab2dd7d) will increase coverage by 2.19%.
Report is 1 commits behind head on master.
The diff coverage is n/a.

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #2531      +/-   ##
============================================
+ Coverage     76.55%   78.74%   +2.19%     
  Complexity      267      267              
============================================
  Files           138      112      -26     
  Lines         17459    13485    -3974     
  Branches        976        0     -976     
============================================
- Hits          13365    10619    -2746     
+ Misses         3574     2866     -708     
+ Partials        520        0     -520     

Flag	Coverage Δ
appsec-extension	?
tracer-extension	78.70% <ø> (ø)
tracer-integrations	79.49% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

see 26 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ab2dd7d...4257a5e. Read the comment docs.