Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement safety guards for TriggerProbes #7843

Merged
merged 3 commits into from
Oct 31, 2024
Merged

Implement safety guards for TriggerProbes #7843

merged 3 commits into from
Oct 31, 2024

Conversation

evanchooly
Copy link
Contributor

@evanchooly evanchooly commented Oct 28, 2024 •
edited
Loading

What Does This Do

This PR introduces rate limiting, probe conditions, cool down support for trigger probes.

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [DEBUG-2953 DEBUG-2954 DEBUG-2720]

@pr-commenter
Copy link

pr-commenter bot commented Oct 28, 2024 •
edited
Loading

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1730404953 1730405330
end_time 2024-10-31T20:03:47 2024-10-31T20:10:05
git_branch master evanchooly/DEBUG-2954
git_commit_sha 16a53f2 cd3beb8
start_time 2024-10-31T20:02:34 2024-10-31T20:08:50
See matching parameters
Baseline Candidate
ci_job_id 692246251 692246251
ci_pipeline_id 48005842 48005842
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1730404220 1730404220

Summary

Found 0 performance improvements and 5 performance regressions! Performance is the same for 1 metrics, 9 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:loop worse
[+608.064µs; +635.503µs] or [+5.963%; +6.232%]		 worse
[+603.437µs; +673.059µs] or [+5.820%; +6.491%]		 worse
[+552.499µs; +683.223µs] or [+5.277%; +6.526%]		 worse
[+349.794µs; +717.547µs] or [+3.209%; +6.582%]		 worse
[-6.630op/s; -4.474op/s] or [-6.961%; -4.698%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-31.481µs; +85.383µs] or [-12.216%; +33.132%]		 unstable
[-35.659µs; +113.354µs] or [-12.233%; +38.886%]		 unstable
[-41.981µs; +134.246µs] or [-13.786%; +44.085%]		 unstable
[-408.969µs; +1251.960µs] or [-68.479%; +209.632%]		 unstable
[-215.003op/s; +72.756op/s] or [-7.955%; +2.692%]
scenario:basic same unstable
[-10.965µs; +35.982µs] or [-3.685%; +12.094%]		 unstable
[-14.867µs; +49.075µs] or [-4.856%; +16.030%]		 unstable
[-281.776µs; +851.031µs] or [-51.917%; +156.801%]		 unstable
[-231.591op/s; +96.638op/s] or [-8.800%; +3.672%]
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (291.505 µs) : 266, 317
.   : milestone, 292,
basic (297.513 µs) : 287, 308
.   : milestone, 298,
loop (10.369 ms) : 10343, 10394
.   : milestone, 10369,
section candidate
noprobe (330.352 µs) : 233, 428
.   : milestone, 330,
basic (310.021 µs) : 279, 341
.   : milestone, 310,
loop (11.007 ms) : 10969, 11045
.   : milestone, 11007,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 291.505 µs [266.111 µs, 316.899 µs]
basic 297.513 µs [287.196 µs, 307.829 µs]
loop 10.369 ms [10.343 ms, 10.394 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 330.352 µs [233.139 µs, 427.566 µs]
basic 310.021 µs [279.224 µs, 340.819 µs]
loop 11.007 ms [10.969 ms, 11.045 ms]

@pr-commenter
Copy link

pr-commenter bot commented Oct 28, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master evanchooly/DEBUG-2954
git_commit_date 1730396534 1730404220
git_commit_sha 16a53f2 cd3beb8
release_version 1.42.0-SNAPSHOT~16a53f235c 1.42.0-SNAPSHOT~cd3beb8b0c
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1730406784 1730406784
ci_job_id 692246245 692246245
ci_pipeline_id 48005842 48005842
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 55 metrics, 8 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.42.0-SNAPSHOT~cd3beb8b0c, baseline=1.42.0-SNAPSHOT~16a53f235c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.082 s) : 0, 1082028
Total [baseline] (8.579 s) : 0, 8578515
Agent [candidate] (1.075 s) : 0, 1075046
Total [candidate] (8.549 s) : 0, 8549299
section iast
Agent [baseline] (1.202 s) : 0, 1202145
Total [baseline] (9.109 s) : 0, 9109367
Agent [candidate] (1.202 s) : 0, 1201580
Total [candidate] (9.134 s) : 0, 9134285
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.203 s) : 0, 1202833
Total [baseline] (9.074 s) : 0, 9073521
Agent [candidate] (1.203 s) : 0, 1202725
Total [candidate] (9.127 s) : 0, 9127434
section iast_TELEMETRY_OFF
Agent [baseline] (1.201 s) : 0, 1201400
Total [baseline] (9.158 s) : 0, 9157800
Agent [candidate] (1.2 s) : 0, 1200487
Total [candidate] (9.112 s) : 0, 9112253
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.082 s -
Agent iast 1.202 s 120.117 ms (11.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.203 s 120.806 ms (11.2%)
Agent iast_TELEMETRY_OFF 1.201 s 119.373 ms (11.0%)
Total tracing 8.579 s -
Total iast 9.109 s 530.851 ms (6.2%)
Total iast_HARDCODED_SECRET_DISABLED 9.074 s 495.005 ms (5.8%)
Total iast_TELEMETRY_OFF 9.158 s 579.285 ms (6.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.075 s -
Agent iast 1.202 s 126.534 ms (11.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.203 s 127.678 ms (11.9%)
Agent iast_TELEMETRY_OFF 1.2 s 125.441 ms (11.7%)
Total tracing 8.549 s -
Total iast 9.134 s 584.987 ms (6.8%)
Total iast_HARDCODED_SECRET_DISABLED 9.127 s 578.135 ms (6.8%)
Total iast_TELEMETRY_OFF 9.112 s 562.954 ms (6.6%) 
gantt
    title insecure-bank - break down per module: candidate=1.42.0-SNAPSHOT~cd3beb8b0c, baseline=1.42.0-SNAPSHOT~16a53f235c

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (687.786 ms) : 0, 687786
BytebuddyAgent [candidate] (683.573 ms) : 0, 683573
GlobalTracer [baseline] (315.228 ms) : 0, 315228
GlobalTracer [candidate] (313.628 ms) : 0, 313628
AppSec [baseline] (54.096 ms) : 0, 54096
AppSec [candidate] (53.855 ms) : 0, 53855
Remote Config [baseline] (674.796 µs) : 0, 675
Remote Config [candidate] (666.093 µs) : 0, 666
Telemetry [baseline] (10.533 ms) : 0, 10533
Telemetry [candidate] (9.739 ms) : 0, 9739
section iast
BytebuddyAgent [baseline] (800.442 ms) : 0, 800442
BytebuddyAgent [candidate] (800.594 ms) : 0, 800594
GlobalTracer [baseline] (303.112 ms) : 0, 303112
GlobalTracer [candidate] (302.048 ms) : 0, 302048
AppSec [baseline] (56.785 ms) : 0, 56785
AppSec [candidate] (56.523 ms) : 0, 56523
IAST [baseline] (20.248 ms) : 0, 20248
IAST [candidate] (20.722 ms) : 0, 20722
Remote Config [baseline] (595.926 µs) : 0, 596
Remote Config [candidate] (611.907 µs) : 0, 612
Telemetry [baseline] (7.36 ms) : 0, 7360
Telemetry [candidate] (7.452 ms) : 0, 7452
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (800.477 ms) : 0, 800477
BytebuddyAgent [candidate] (800.522 ms) : 0, 800522
GlobalTracer [baseline] (303.175 ms) : 0, 303175
GlobalTracer [candidate] (303.03 ms) : 0, 303030
AppSec [baseline] (57.293 ms) : 0, 57293
AppSec [candidate] (57.478 ms) : 0, 57478
IAST [baseline] (20.26 ms) : 0, 20260
IAST [candidate] (19.921 ms) : 0, 19921
Remote Config [baseline] (599.211 µs) : 0, 599
Remote Config [candidate] (622.623 µs) : 0, 623
Telemetry [baseline] (7.429 ms) : 0, 7429
Telemetry [candidate] (7.52 ms) : 0, 7520
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (798.688 ms) : 0, 798688
BytebuddyAgent [candidate] (798.624 ms) : 0, 798624
GlobalTracer [baseline] (303.671 ms) : 0, 303671
GlobalTracer [candidate] (303.234 ms) : 0, 303234
AppSec [baseline] (57.466 ms) : 0, 57466
AppSec [candidate] (56.793 ms) : 0, 56793
IAST [baseline] (20.015 ms) : 0, 20015
IAST [candidate] (20.279 ms) : 0, 20279
Remote Config [baseline] (602.636 µs) : 0, 603
Remote Config [candidate] (590.783 µs) : 0, 591
Telemetry [baseline] (7.345 ms) : 0, 7345
Telemetry [candidate] (7.32 ms) : 0, 7320
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.42.0-SNAPSHOT~cd3beb8b0c, baseline=1.42.0-SNAPSHOT~16a53f235c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.087 s) : 0, 1087460
Total [baseline] (10.439 s) : 0, 10439001
Agent [candidate] (1.086 s) : 0, 1086442
Total [candidate] (10.472 s) : 0, 10471536
section appsec
Agent [baseline] (1.212 s) : 0, 1212008
Total [baseline] (10.675 s) : 0, 10674547
Agent [candidate] (1.215 s) : 0, 1215355
Total [candidate] (10.68 s) : 0, 10679623
section iast
Agent [baseline] (1.204 s) : 0, 1204134
Total [baseline] (10.911 s) : 0, 10911075
Agent [candidate] (1.21 s) : 0, 1209649
Total [candidate] (10.911 s) : 0, 10910696
section profiling
Agent [baseline] (1.284 s) : 0, 1283885
Total [baseline] (10.688 s) : 0, 10687942
Agent [candidate] (1.278 s) : 0, 1277766
Total [candidate] (10.677 s) : 0, 10676663
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.087 s -
Agent appsec 1.212 s 124.548 ms (11.5%)
Agent iast 1.204 s 116.674 ms (10.7%)
Agent profiling 1.284 s 196.425 ms (18.1%)
Total tracing 10.439 s -
Total appsec 10.675 s 235.546 ms (2.3%)
Total iast 10.911 s 472.074 ms (4.5%)
Total profiling 10.688 s 248.942 ms (2.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.086 s -
Agent appsec 1.215 s 128.913 ms (11.9%)
Agent iast 1.21 s 123.207 ms (11.3%)
Agent profiling 1.278 s 191.324 ms (17.6%)
Total tracing 10.472 s -
Total appsec 10.68 s 208.087 ms (2.0%)
Total iast 10.911 s 439.16 ms (4.2%)
Total profiling 10.677 s 205.127 ms (2.0%) 
gantt
    title petclinic - break down per module: candidate=1.42.0-SNAPSHOT~cd3beb8b0c, baseline=1.42.0-SNAPSHOT~16a53f235c

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (691.347 ms) : 0, 691347
BytebuddyAgent [candidate] (690.126 ms) : 0, 690126
GlobalTracer [baseline] (316.764 ms) : 0, 316764
GlobalTracer [candidate] (315.759 ms) : 0, 315759
AppSec [baseline] (54.413 ms) : 0, 54413
AppSec [candidate] (54.215 ms) : 0, 54215
Remote Config [baseline] (665.282 µs) : 0, 665
Remote Config [candidate] (665.493 µs) : 0, 665
Telemetry [baseline] (10.557 ms) : 0, 10557
Telemetry [candidate] (11.952 ms) : 0, 11952
section appsec
BytebuddyAgent [baseline] (702.262 ms) : 0, 702262
BytebuddyAgent [candidate] (706.268 ms) : 0, 706268
GlobalTracer [baseline] (312.325 ms) : 0, 312325
GlobalTracer [candidate] (311.689 ms) : 0, 311689
AppSec [baseline] (165.851 ms) : 0, 165851
AppSec [candidate] (164.519 ms) : 0, 164519
Remote Config [baseline] (628.431 µs) : 0, 628
Remote Config [candidate] (646.741 µs) : 0, 647
Telemetry [baseline] (8.116 ms) : 0, 8116
Telemetry [candidate] (8.448 ms) : 0, 8448
IAST [baseline] (18.458 ms) : 0, 18458
IAST [candidate] (20.185 ms) : 0, 20185
section iast
BytebuddyAgent [baseline] (801.009 ms) : 0, 801009
BytebuddyAgent [candidate] (806.328 ms) : 0, 806328
GlobalTracer [baseline] (304.189 ms) : 0, 304189
GlobalTracer [candidate] (304.276 ms) : 0, 304276
AppSec [baseline] (57.28 ms) : 0, 57280
AppSec [candidate] (57.384 ms) : 0, 57384
Remote Config [baseline] (581.581 µs) : 0, 582
Remote Config [candidate] (603.4 µs) : 0, 603
Telemetry [baseline] (7.381 ms) : 0, 7381
Telemetry [candidate] (7.397 ms) : 0, 7397
IAST [baseline] (20.067 ms) : 0, 20067
IAST [candidate] (19.948 ms) : 0, 19948
section profiling
ProfilingAgent [baseline] (90.653 ms) : 0, 90653
ProfilingAgent [candidate] (91.727 ms) : 0, 91727
BytebuddyAgent [baseline] (683.673 ms) : 0, 683673
BytebuddyAgent [candidate] (683.223 ms) : 0, 683223
GlobalTracer [baseline] (401.959 ms) : 0, 401959
GlobalTracer [candidate] (398.196 ms) : 0, 398196
AppSec [baseline] (54.795 ms) : 0, 54795
AppSec [candidate] (54.104 ms) : 0, 54104
Remote Config [baseline] (689.539 µs) : 0, 690
Remote Config [candidate] (656.285 µs) : 0, 656
Telemetry [baseline] (13.246 ms) : 0, 13246
Telemetry [candidate] (11.047 ms) : 0, 11047
Profiling [baseline] (90.676 ms) : 0, 90676
Profiling [candidate] (91.75 ms) : 0, 91750
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-10-31T20:01:26 2024-10-31T20:08:21
git_branch master evanchooly/DEBUG-2954
git_commit_date 1730396534 1730404220
git_commit_sha 16a53f2 cd3beb8
release_version 1.42.0-SNAPSHOT~16a53f235c 1.42.0-SNAPSHOT~cd3beb8b0c
start_time 2024-10-31T20:01:12 2024-10-31T20:08:08
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1730405652 1730405652
ci_job_id 692246246 692246246
ci_pipeline_id 48005842 48005842
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.42.0-SNAPSHOT~cd3beb8b0c, baseline=1.42.0-SNAPSHOT~16a53f235c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.351 ms) : 1331, 1370
.   : milestone, 1351,
appsec (1.735 ms) : 1713, 1758
.   : milestone, 1735,
appsec_no_iast (1.718 ms) : 1693, 1743
.   : milestone, 1718,
iast (1.472 ms) : 1449, 1496
.   : milestone, 1472,
profiling (1.517 ms) : 1493, 1541
.   : milestone, 1517,
tracing (1.494 ms) : 1469, 1518
.   : milestone, 1494,
section candidate
no_agent (1.348 ms) : 1328, 1368
.   : milestone, 1348,
appsec (1.734 ms) : 1710, 1758
.   : milestone, 1734,
appsec_no_iast (1.737 ms) : 1713, 1762
.   : milestone, 1737,
iast (1.493 ms) : 1469, 1516
.   : milestone, 1493,
profiling (1.496 ms) : 1471, 1520
.   : milestone, 1496,
tracing (1.471 ms) : 1446, 1496
.   : milestone, 1471,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.351 ms [1.331 ms, 1.37 ms] -
appsec 1.735 ms [1.713 ms, 1.758 ms] 384.421 µs (28.5%)
appsec_no_iast 1.718 ms [1.693 ms, 1.743 ms] 366.805 µs (27.2%)
iast 1.472 ms [1.449 ms, 1.496 ms] 121.505 µs (9.0%)
profiling 1.517 ms [1.493 ms, 1.541 ms] 166.118 µs (12.3%)
tracing 1.494 ms [1.469 ms, 1.518 ms] 142.744 µs (10.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.348 ms [1.328 ms, 1.368 ms] -
appsec 1.734 ms [1.71 ms, 1.758 ms] 385.638 µs (28.6%)
appsec_no_iast 1.737 ms [1.713 ms, 1.762 ms] 389.275 µs (28.9%)
iast 1.493 ms [1.469 ms, 1.516 ms] 144.498 µs (10.7%)
profiling 1.496 ms [1.471 ms, 1.52 ms] 147.454 µs (10.9%)
tracing 1.471 ms [1.446 ms, 1.496 ms] 123.083 µs (9.1%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.42.0-SNAPSHOT~cd3beb8b0c, baseline=1.42.0-SNAPSHOT~16a53f235c
    dateFormat X
    axisFormat %s
section baseline
no_agent (374.264 µs) : 355, 394
.   : milestone, 374,
iast (488.916 µs) : 467, 510
.   : milestone, 489,
iast_FULL (642.939 µs) : 622, 664
.   : milestone, 643,
iast_GLOBAL (524.021 µs) : 501, 547
.   : milestone, 524,
iast_HARDCODED_SECRET_DISABLED (488.844 µs) : 467, 510
.   : milestone, 489,
iast_INACTIVE (450.29 µs) : 429, 472
.   : milestone, 450,
iast_TELEMETRY_OFF (476.849 µs) : 456, 498
.   : milestone, 477,
tracing (440.77 µs) : 421, 461
.   : milestone, 441,
section candidate
no_agent (366.448 µs) : 346, 386
.   : milestone, 366,
iast (483.241 µs) : 462, 504
.   : milestone, 483,
iast_FULL (650.282 µs) : 629, 672
.   : milestone, 650,
iast_GLOBAL (531.607 µs) : 509, 554
.   : milestone, 532,
iast_HARDCODED_SECRET_DISABLED (486.093 µs) : 465, 507
.   : milestone, 486,
iast_INACTIVE (451.331 µs) : 430, 472
.   : milestone, 451,
iast_TELEMETRY_OFF (476.368 µs) : 455, 498
.   : milestone, 476,
tracing (444.799 µs) : 424, 465
.   : milestone, 445,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 374.264 µs [354.605 µs, 393.922 µs] -
iast 488.916 µs [467.492 µs, 510.339 µs] 114.652 µs (30.6%)
iast_FULL 642.939 µs [621.707 µs, 664.17 µs] 268.675 µs (71.8%)
iast_GLOBAL 524.021 µs [501.273 µs, 546.768 µs] 149.757 µs (40.0%)
iast_HARDCODED_SECRET_DISABLED 488.844 µs [467.366 µs, 510.322 µs] 114.58 µs (30.6%)
iast_INACTIVE 450.29 µs [429.056 µs, 471.525 µs] 76.026 µs (20.3%)
iast_TELEMETRY_OFF 476.849 µs [455.769 µs, 497.929 µs] 102.585 µs (27.4%)
tracing 440.77 µs [420.795 µs, 460.746 µs] 66.507 µs (17.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 366.448 µs [346.418 µs, 386.477 µs] -
iast 483.241 µs [462.305 µs, 504.177 µs] 116.793 µs (31.9%)
iast_FULL 650.282 µs [628.909 µs, 671.655 µs] 283.835 µs (77.5%)
iast_GLOBAL 531.607 µs [508.813 µs, 554.401 µs] 165.159 µs (45.1%)
iast_HARDCODED_SECRET_DISABLED 486.093 µs [464.99 µs, 507.196 µs] 119.646 µs (32.7%)
iast_INACTIVE 451.331 µs [430.346 µs, 472.317 µs] 84.884 µs (23.2%)
iast_TELEMETRY_OFF 476.368 µs [455.226 µs, 497.51 µs] 109.92 µs (30.0%)
tracing 444.799 µs [424.367 µs, 465.231 µs] 78.351 µs (21.4%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master evanchooly/DEBUG-2954
git_commit_date 1730396534 1730404220
git_commit_sha 16a53f2 cd3beb8
release_version 1.42.0-SNAPSHOT~16a53f235c 1.42.0-SNAPSHOT~cd3beb8b0c
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1730406304 1730406304
ci_job_id 692246247 692246247
ci_pipeline_id 48005842 48005842
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.42.0-SNAPSHOT~cd3beb8b0c, baseline=1.42.0-SNAPSHOT~16a53f235c
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.036 s) : 15036000, 15036000
.   : milestone, 15036000,
appsec (15.257 s) : 15257000, 15257000
.   : milestone, 15257000,
iast (19.121 s) : 19121000, 19121000
.   : milestone, 19121000,
iast_GLOBAL (18.32 s) : 18320000, 18320000
.   : milestone, 18320000,
profiling (15.43 s) : 15430000, 15430000
.   : milestone, 15430000,
tracing (14.87 s) : 14870000, 14870000
.   : milestone, 14870000,
section candidate
no_agent (15.405 s) : 15405000, 15405000
.   : milestone, 15405000,
appsec (14.973 s) : 14973000, 14973000
.   : milestone, 14973000,
iast (18.905 s) : 18905000, 18905000
.   : milestone, 18905000,
iast_GLOBAL (18.074 s) : 18074000, 18074000
.   : milestone, 18074000,
profiling (15.727 s) : 15727000, 15727000
.   : milestone, 15727000,
tracing (15.061 s) : 15061000, 15061000
.   : milestone, 15061000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.036 s [15.036 s, 15.036 s] -
appsec 15.257 s [15.257 s, 15.257 s] 221.0 ms (1.5%)
iast 19.121 s [19.121 s, 19.121 s] 4.085 s (27.2%)
iast_GLOBAL 18.32 s [18.32 s, 18.32 s] 3.284 s (21.8%)
profiling 15.43 s [15.43 s, 15.43 s] 394.0 ms (2.6%)
tracing 14.87 s [14.87 s, 14.87 s] -166.0 ms (-1.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.405 s [15.405 s, 15.405 s] -
appsec 14.973 s [14.973 s, 14.973 s] -432.0 ms (-2.8%)
iast 18.905 s [18.905 s, 18.905 s] 3.5 s (22.7%)
iast_GLOBAL 18.074 s [18.074 s, 18.074 s] 2.669 s (17.3%)
profiling 15.727 s [15.727 s, 15.727 s] 322.0 ms (2.1%)
tracing 15.061 s [15.061 s, 15.061 s] -344.0 ms (-2.2%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.42.0-SNAPSHOT~cd3beb8b0c, baseline=1.42.0-SNAPSHOT~16a53f235c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.468 ms) : 1457, 1480
.   : milestone, 1468,
appsec (2.334 ms) : 2293, 2375
.   : milestone, 2334,
iast (2.076 ms) : 2024, 2128
.   : milestone, 2076,
iast_GLOBAL (2.118 ms) : 2066, 2170
.   : milestone, 2118,
profiling (1.954 ms) : 1911, 1997
.   : milestone, 1954,
tracing (1.924 ms) : 1884, 1963
.   : milestone, 1924,
section candidate
no_agent (1.469 ms) : 1458, 1481
.   : milestone, 1469,
appsec (2.34 ms) : 2299, 2381
.   : milestone, 2340,
iast (2.074 ms) : 2021, 2126
.   : milestone, 2074,
iast_GLOBAL (2.126 ms) : 2074, 2179
.   : milestone, 2126,
profiling (1.943 ms) : 1902, 1985
.   : milestone, 1943,
tracing (1.923 ms) : 1883, 1963
.   : milestone, 1923,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.468 ms [1.457 ms, 1.48 ms] -
appsec 2.334 ms [2.293 ms, 2.375 ms] 865.833 µs (59.0%)
iast 2.076 ms [2.024 ms, 2.128 ms] 607.635 µs (41.4%)
iast_GLOBAL 2.118 ms [2.066 ms, 2.17 ms] 649.681 µs (44.2%)
profiling 1.954 ms [1.911 ms, 1.997 ms] 485.736 µs (33.1%)
tracing 1.924 ms [1.884 ms, 1.963 ms] 455.251 µs (31.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.469 ms [1.458 ms, 1.481 ms] -
appsec 2.34 ms [2.299 ms, 2.381 ms] 870.726 µs (59.3%)
iast 2.074 ms [2.021 ms, 2.126 ms] 604.464 µs (41.1%)
iast_GLOBAL 2.126 ms [2.074 ms, 2.179 ms] 657.376 µs (44.7%)
profiling 1.943 ms [1.902 ms, 1.985 ms] 474.36 µs (32.3%)
tracing 1.923 ms [1.883 ms, 1.963 ms] 454.155 µs (30.9%)

@evanchooly evanchooly marked this pull request as ready for review October 29, 2024 13:55
@evanchooly evanchooly requested a review from a team as a code owner October 29, 2024 13:55
@evanchooly evanchooly requested review from cimi and jpbempel and removed request for a team October 29, 2024 13:55
@evanchooly evanchooly marked this pull request as draft October 29, 2024 13:55
@evanchooly evanchooly marked this pull request as ready for review October 29, 2024 14:00
@evanchooly evanchooly force-pushed the evanchooly/DEBUG-2954 branch from 270d95f to e31cf97 Compare October 30, 2024 17:20
@evanchooly
reinstate LogProbe.Sampling
0d7b0a3 
update Sampling to new schema from idan
@evanchooly
more tests for jacoco
cd3beb8 
API clean up
@evanchooly evanchooly merged commit 6bf7492 into master Oct 31, 2024
102 checks passed
@evanchooly evanchooly deleted the evanchooly/DEBUG-2954 branch October 31, 2024 21:04
@github-actions github-actions bot added this to the 1.42.0 milestone Oct 31, 2024
@jordan-wong jordan-wong added comp: debugger Dynamic Instrumentation tag: no release notes Changes to exclude from release notes and removed comp: debugger Dynamic Instrumentation labels Nov 4, 2024
@jpbempel jpbempel added the comp: debugger Dynamic Instrumentation label Nov 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@datadog-datadog-prod-us1 datadog-datadog-prod-us1[bot] datadog-datadog-prod-us1[bot] left review comments

@jpbempel jpbempel jpbempel approved these changes

@cimi cimi Awaiting requested review from cimi cimi is a code owner automatically assigned from DataDog/debugger-java

Assignees
No one assigned
Labels
comp: debugger Dynamic Instrumentation tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.42.0
Development

Successfully merging this pull request may close these issues.
3 participants
@evanchooly @jpbempel @jordan-wong