Update metrics: appsec.waf.requests #8353

Mariovido · 2025-02-07T11:22:18Z

What Does This Do

This adds a new value to some metrics which is necessary for the consolidation of ASM Span Tags, Metrics, and Logs across all supported languages. The newly value will be implemented in the following metrics:

appsec.waf.requests:
- waf_error: If WAF has failed (except timeout)
- block_failure: If the block has failed
- request_excluded: Wheter the request was filtered by an exclusion
- rate_limited: Wheter the trace has been rate limited
- input_truncated: Wheter the data provided to the libddwaf has been truncated

Motivation

Our goal is to implement all the missing ASM Span Tags, Metrics, and Logs.

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: APPSEC-56676

pr-commenter · 2025-02-07T12:03:03Z

Benchmarks

Startup

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-02-07T14:12:11	2025-02-07T14:19:17
git_branch	master	mario.vidal/update_metrics_appsec_request
git_commit_date	1738853998	1738936750
git_commit_sha	`8a74e85`	`d4c5ff2`
release_version	1.47.0-SNAPSHOT~8a74e85918	1.47.0-SNAPSHOT~d4c5ff2d68
start_time	2025-02-07T14:11:57	2025-02-07T14:19:03

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1738938317	1738938317
ci_job_id	798977720	798977720
ci_pipeline_id	55247398	55247398
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-ukzeafmu-project-304-concurrent-0-07gckctr 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-ukzeafmu-project-304-concurrent-0-07gckctr 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 15 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~d4c5ff2d68, baseline=1.47.0-SNAPSHOT~8a74e85918
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.357 ms) : 1337, 1377
.   : milestone, 1357,
appsec (1.76 ms) : 1736, 1784
.   : milestone, 1760,
appsec_no_iast (1.747 ms) : 1723, 1770
.   : milestone, 1747,
iast (1.526 ms) : 1502, 1551
.   : milestone, 1526,
profiling (1.572 ms) : 1547, 1597
.   : milestone, 1572,
tracing (1.514 ms) : 1490, 1538
.   : milestone, 1514,
section candidate
no_agent (1.353 ms) : 1333, 1372
.   : milestone, 1353,
appsec (1.745 ms) : 1721, 1768
.   : milestone, 1745,
appsec_no_iast (1.764 ms) : 1740, 1788
.   : milestone, 1764,
iast (1.495 ms) : 1470, 1520
.   : milestone, 1495,
profiling (1.575 ms) : 1551, 1599
.   : milestone, 1575,
tracing (1.495 ms) : 1470, 1520
.   : milestone, 1495,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.357 ms [1.337 ms, 1.377 ms]	-
appsec	1.76 ms [1.736 ms, 1.784 ms]	402.922 µs (29.7%)
appsec_no_iast	1.747 ms [1.723 ms, 1.77 ms]	389.199 µs (28.7%)
iast	1.526 ms [1.502 ms, 1.551 ms]	168.943 µs (12.4%)
profiling	1.572 ms [1.547 ms, 1.597 ms]	214.946 µs (15.8%)
tracing	1.514 ms [1.49 ms, 1.538 ms]	157.001 µs (11.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.353 ms [1.333 ms, 1.372 ms]	-
appsec	1.745 ms [1.721 ms, 1.768 ms]	391.981 µs (29.0%)
appsec_no_iast	1.764 ms [1.74 ms, 1.788 ms]	410.656 µs (30.4%)
iast	1.495 ms [1.47 ms, 1.52 ms]	141.911 µs (10.5%)
profiling	1.575 ms [1.551 ms, 1.599 ms]	222.121 µs (16.4%)
tracing	1.495 ms [1.47 ms, 1.52 ms]	142.149 µs (10.5%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~d4c5ff2d68, baseline=1.47.0-SNAPSHOT~8a74e85918
    dateFormat X
    axisFormat %s
section baseline
no_agent (382.612 µs) : 362, 404
.   : milestone, 383,
iast (516.044 µs) : 494, 538
.   : milestone, 516,
iast_FULL (746.896 µs) : 725, 769
.   : milestone, 747,
iast_GLOBAL (565.71 µs) : 544, 588
.   : milestone, 566,
iast_HARDCODED_SECRET_DISABLED (516.139 µs) : 495, 538
.   : milestone, 516,
iast_INACTIVE (468.354 µs) : 447, 490
.   : milestone, 468,
iast_TELEMETRY_OFF (495.438 µs) : 473, 518
.   : milestone, 495,
tracing (456.262 µs) : 435, 477
.   : milestone, 456,
section candidate
no_agent (382.256 µs) : 361, 403
.   : milestone, 382,
iast (511.081 µs) : 490, 533
.   : milestone, 511,
iast_FULL (748.896 µs) : 727, 771
.   : milestone, 749,
iast_GLOBAL (551.201 µs) : 530, 573
.   : milestone, 551,
iast_HARDCODED_SECRET_DISABLED (510.255 µs) : 489, 532
.   : milestone, 510,
iast_INACTIVE (466.94 µs) : 446, 488
.   : milestone, 467,
iast_TELEMETRY_OFF (501.705 µs) : 480, 523
.   : milestone, 502,
tracing (459.624 µs) : 439, 481
.   : milestone, 460,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	382.612 µs [361.521 µs, 403.703 µs]	-
iast	516.044 µs [494.448 µs, 537.64 µs]	133.432 µs (34.9%)
iast_FULL	746.896 µs [725.138 µs, 768.654 µs]	364.284 µs (95.2%)
iast_GLOBAL	565.71 µs [543.766 µs, 587.655 µs]	183.098 µs (47.9%)
iast_HARDCODED_SECRET_DISABLED	516.139 µs [494.666 µs, 537.611 µs]	133.527 µs (34.9%)
iast_INACTIVE	468.354 µs [446.946 µs, 489.763 µs]	85.742 µs (22.4%)
iast_TELEMETRY_OFF	495.438 µs [473.269 µs, 517.606 µs]	112.826 µs (29.5%)
tracing	456.262 µs [435.04 µs, 477.484 µs]	73.65 µs (19.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	382.256 µs [361.295 µs, 403.217 µs]	-
iast	511.081 µs [489.601 µs, 532.561 µs]	128.825 µs (33.7%)
iast_FULL	748.896 µs [726.725 µs, 771.066 µs]	366.64 µs (95.9%)
iast_GLOBAL	551.201 µs [529.606 µs, 572.796 µs]	168.946 µs (44.2%)
iast_HARDCODED_SECRET_DISABLED	510.255 µs [488.733 µs, 531.777 µs]	127.999 µs (33.5%)
iast_INACTIVE	466.94 µs [445.824 µs, 488.056 µs]	84.685 µs (22.2%)
iast_TELEMETRY_OFF	501.705 µs [480.266 µs, 523.144 µs]	119.449 µs (31.2%)
tracing	459.624 µs [438.526 µs, 480.722 µs]	77.369 µs (20.2%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mario.vidal/update_metrics_appsec_request
git_commit_date	1738853998	1738936750
git_commit_sha	`8a74e85`	`d4c5ff2`
release_version	1.47.0-SNAPSHOT~8a74e85918	1.47.0-SNAPSHOT~d4c5ff2d68

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1738938768	1738938768
ci_job_id	798977722	798977722
ci_pipeline_id	55247398	55247398
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-m8adyusn-project-304-concurrent-0-fl1u9qlu 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-m8adyusn-project-304-concurrent-0-fl1u9qlu 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~d4c5ff2d68, baseline=1.47.0-SNAPSHOT~8a74e85918
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.703 s) : 14703000, 14703000
.   : milestone, 14703000,
appsec (14.931 s) : 14931000, 14931000
.   : milestone, 14931000,
iast (18.902 s) : 18902000, 18902000
.   : milestone, 18902000,
iast_GLOBAL (17.764 s) : 17764000, 17764000
.   : milestone, 17764000,
profiling (15.157 s) : 15157000, 15157000
.   : milestone, 15157000,
tracing (15.235 s) : 15235000, 15235000
.   : milestone, 15235000,
section candidate
no_agent (15.594 s) : 15594000, 15594000
.   : milestone, 15594000,
appsec (15.058 s) : 15058000, 15058000
.   : milestone, 15058000,
iast (18.514 s) : 18514000, 18514000
.   : milestone, 18514000,
iast_GLOBAL (17.934 s) : 17934000, 17934000
.   : milestone, 17934000,
profiling (15.033 s) : 15033000, 15033000
.   : milestone, 15033000,
tracing (15.127 s) : 15127000, 15127000
.   : milestone, 15127000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.703 s [14.703 s, 14.703 s]	-
appsec	14.931 s [14.931 s, 14.931 s]	228.0 ms (1.6%)
iast	18.902 s [18.902 s, 18.902 s]	4.199 s (28.6%)
iast_GLOBAL	17.764 s [17.764 s, 17.764 s]	3.061 s (20.8%)
profiling	15.157 s [15.157 s, 15.157 s]	454.0 ms (3.1%)
tracing	15.235 s [15.235 s, 15.235 s]	532.0 ms (3.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.594 s [15.594 s, 15.594 s]	-
appsec	15.058 s [15.058 s, 15.058 s]	-536.0 ms (-3.4%)
iast	18.514 s [18.514 s, 18.514 s]	2.92 s (18.7%)
iast_GLOBAL	17.934 s [17.934 s, 17.934 s]	2.34 s (15.0%)
profiling	15.033 s [15.033 s, 15.033 s]	-561.0 ms (-3.6%)
tracing	15.127 s [15.127 s, 15.127 s]	-467.0 ms (-3.0%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~d4c5ff2d68, baseline=1.47.0-SNAPSHOT~8a74e85918
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.467 ms) : 1455, 1478
.   : milestone, 1467,
appsec (2.349 ms) : 2306, 2393
.   : milestone, 2349,
iast (2.101 ms) : 2046, 2156
.   : milestone, 2101,
iast_GLOBAL (2.145 ms) : 2089, 2200
.   : milestone, 2145,
profiling (1.966 ms) : 1922, 2009
.   : milestone, 1966,
tracing (1.933 ms) : 1891, 1975
.   : milestone, 1933,
section candidate
no_agent (1.47 ms) : 1458, 1481
.   : milestone, 1470,
appsec (2.338 ms) : 2295, 2381
.   : milestone, 2338,
iast (2.102 ms) : 2048, 2157
.   : milestone, 2102,
iast_GLOBAL (2.14 ms) : 2085, 2196
.   : milestone, 2140,
profiling (1.954 ms) : 1911, 1997
.   : milestone, 1954,
tracing (1.946 ms) : 1903, 1989
.   : milestone, 1946,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.467 ms [1.455 ms, 1.478 ms]	-
appsec	2.349 ms [2.306 ms, 2.393 ms]	882.574 µs (60.2%)
iast	2.101 ms [2.046 ms, 2.156 ms]	634.388 µs (43.3%)
iast_GLOBAL	2.145 ms [2.089 ms, 2.2 ms]	678.042 µs (46.2%)
profiling	1.966 ms [1.922 ms, 2.009 ms]	499.028 µs (34.0%)
tracing	1.933 ms [1.891 ms, 1.975 ms]	466.736 µs (31.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.47 ms [1.458 ms, 1.481 ms]	-
appsec	2.338 ms [2.295 ms, 2.381 ms]	867.763 µs (59.0%)
iast	2.102 ms [2.048 ms, 2.157 ms]	632.425 µs (43.0%)
iast_GLOBAL	2.14 ms [2.085 ms, 2.196 ms]	670.364 µs (45.6%)
profiling	1.954 ms [1.911 ms, 1.997 ms]	484.059 µs (32.9%)
tracing	1.946 ms [1.903 ms, 1.989 ms]	476.037 µs (32.4%)

Add waf_error tag to appsec.waf.requests metric

41779c2

Mariovido added type: enhancement comp: asm waf Application Security Management (WAF) comp: telemetry Telemetry labels Feb 7, 2025

Mariovido added 2 commits February 7, 2025 14:28

Add tag rate_limited

2ad2645

Add tag block_failure

d4c5ff2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update metrics: appsec.waf.requests #8353

Update metrics: appsec.waf.requests #8353

Mariovido commented Feb 7, 2025 •

edited

Loading

pr-commenter bot commented Feb 7, 2025 •

edited

Loading

Update metrics: appsec.waf.requests #8353

Are you sure you want to change the base?

Update metrics: appsec.waf.requests #8353

Conversation

Mariovido commented Feb 7, 2025 • edited Loading

What Does This Do

Motivation

Additional Notes

Contributor Checklist

pr-commenter bot commented Feb 7, 2025 • edited Loading

Benchmarks

Startup

Load

Parameters

Summary

Dacapo

Parameters

Summary

Mariovido commented Feb 7, 2025 •

edited

Loading

pr-commenter bot commented Feb 7, 2025 •

edited

Loading