Skip to content

Add Spring Boot 3.3 smoke tests #7932

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Nov 12, 2024
Merged

Add Spring Boot 3.3 smoke tests #7932

merged 1 commit into from
Nov 12, 2024

Conversation

smola
Copy link
Member

@smola smola commented Nov 11, 2024

What Does This Do

Add Spring Boot 3.3 smoke tests, based on Spring Boot 3.0.

Motivation

To help testing #7931, but this is of more broad interest, given past escalations specific to Spring Boot 3.2 or later.

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@smola smola added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Nov 11, 2024
@smola smola requested a review from a team as a code owner November 11, 2024 17:28
@smola smola requested a review from dougqh November 11, 2024 17:28
@smola smola force-pushed the smola/spring-boot-3.3-smoke-tests branch from 314c8b1 to 7d2a924 Compare November 11, 2024 17:43
@pr-commenter
Copy link

pr-commenter bot commented Nov 11, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/spring-boot-3.3-smoke-tests
git_commit_date 1731340376 1731347005
git_commit_sha 31bfaf7 7d2a924
release_version 1.43.0-SNAPSHOT~31bfaf7217 1.43.0-SNAPSHOT~7d2a924120
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1731349419 1731349419
ci_job_id 702530629 702530629
ci_pipeline_id 48724297 48724297
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 53 metrics, 10 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.43.0-SNAPSHOT~7d2a924120, baseline=1.43.0-SNAPSHOT~31bfaf7217

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.086 s) : 0, 1086100
Total [baseline] (8.579 s) : 0, 8578852
Agent [candidate] (1.089 s) : 0, 1089127
Total [candidate] (8.597 s) : 0, 8597313
section iast
Agent [baseline] (1.219 s) : 0, 1218764
Total [baseline] (9.142 s) : 0, 9142173
Agent [candidate] (1.209 s) : 0, 1208737
Total [candidate] (9.137 s) : 0, 9137477
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.217 s) : 0, 1217406
Total [baseline] (9.114 s) : 0, 9113856
Agent [candidate] (1.206 s) : 0, 1206485
Total [candidate] (9.1 s) : 0, 9100047
section iast_TELEMETRY_OFF
Agent [baseline] (1.205 s) : 0, 1205118
Total [baseline] (9.121 s) : 0, 9120866
Agent [candidate] (1.206 s) : 0, 1205735
Total [candidate] (9.113 s) : 0, 9112678
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.086 s -
Agent iast 1.219 s 132.664 ms (12.2%)
Agent iast_HARDCODED_SECRET_DISABLED 1.217 s 131.305 ms (12.1%)
Agent iast_TELEMETRY_OFF 1.205 s 119.018 ms (11.0%)
Total tracing 8.579 s -
Total iast 9.142 s 563.32 ms (6.6%)
Total iast_HARDCODED_SECRET_DISABLED 9.114 s 535.003 ms (6.2%)
Total iast_TELEMETRY_OFF 9.121 s 542.014 ms (6.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.089 s -
Agent iast 1.209 s 119.61 ms (11.0%)
Agent iast_HARDCODED_SECRET_DISABLED 1.206 s 117.359 ms (10.8%)
Agent iast_TELEMETRY_OFF 1.206 s 116.609 ms (10.7%)
Total tracing 8.597 s -
Total iast 9.137 s 540.164 ms (6.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.1 s 502.734 ms (5.8%)
Total iast_TELEMETRY_OFF 9.113 s 515.365 ms (6.0%) 
gantt
    title insecure-bank - break down per module: candidate=1.43.0-SNAPSHOT~7d2a924120, baseline=1.43.0-SNAPSHOT~31bfaf7217

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (690.07 ms) : 0, 690070
BytebuddyAgent [candidate] (692.724 ms) : 0, 692724
GlobalTracer [baseline] (316.789 ms) : 0, 316789
GlobalTracer [candidate] (317.844 ms) : 0, 317844
AppSec [baseline] (54.236 ms) : 0, 54236
AppSec [candidate] (54.617 ms) : 0, 54617
Remote Config [baseline] (685.79 µs) : 0, 686
Remote Config [candidate] (687.643 µs) : 0, 688
Telemetry [baseline] (10.481 ms) : 0, 10481
Telemetry [candidate] (9.318 ms) : 0, 9318
section iast
BytebuddyAgent [baseline] (811.164 ms) : 0, 811164
BytebuddyAgent [candidate] (803.449 ms) : 0, 803449
GlobalTracer [baseline] (306.631 ms) : 0, 306631
GlobalTracer [candidate] (304.948 ms) : 0, 304948
AppSec [baseline] (57.957 ms) : 0, 57957
AppSec [candidate] (56.982 ms) : 0, 56982
IAST [baseline] (20.904 ms) : 0, 20904
IAST [candidate] (21.403 ms) : 0, 21403
Remote Config [baseline] (630.034 µs) : 0, 630
Remote Config [candidate] (620.236 µs) : 0, 620
Telemetry [baseline] (7.55 ms) : 0, 7550
Telemetry [candidate] (7.512 ms) : 0, 7512
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (809.712 ms) : 0, 809712
BytebuddyAgent [candidate] (801.485 ms) : 0, 801485
GlobalTracer [baseline] (306.89 ms) : 0, 306890
GlobalTracer [candidate] (304.548 ms) : 0, 304548
AppSec [baseline] (57.947 ms) : 0, 57947
AppSec [candidate] (57.215 ms) : 0, 57215
IAST [baseline] (20.83 ms) : 0, 20830
IAST [candidate] (21.423 ms) : 0, 21423
Remote Config [baseline] (620.192 µs) : 0, 620
Remote Config [candidate] (608.641 µs) : 0, 609
Telemetry [baseline] (7.518 ms) : 0, 7518
Telemetry [candidate] (7.423 ms) : 0, 7423
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (800.43 ms) : 0, 800430
BytebuddyAgent [candidate] (799.979 ms) : 0, 799979
GlobalTracer [baseline] (304.17 ms) : 0, 304170
GlobalTracer [candidate] (305.096 ms) : 0, 305096
AppSec [baseline] (58.443 ms) : 0, 58443
AppSec [candidate] (57.515 ms) : 0, 57515
IAST [baseline] (20.331 ms) : 0, 20331
IAST [candidate] (21.26 ms) : 0, 21260
Remote Config [baseline] (600.739 µs) : 0, 601
Remote Config [candidate] (609.16 µs) : 0, 609
Telemetry [baseline] (7.388 ms) : 0, 7388
Telemetry [candidate] (7.496 ms) : 0, 7496
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.43.0-SNAPSHOT~7d2a924120, baseline=1.43.0-SNAPSHOT~31bfaf7217

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.081 s) : 0, 1080900
Total [baseline] (10.399 s) : 0, 10398768
Agent [candidate] (1.085 s) : 0, 1085061
Total [candidate] (10.381 s) : 0, 10381117
section appsec
Agent [baseline] (1.216 s) : 0, 1215510
Total [baseline] (10.677 s) : 0, 10676654
Agent [candidate] (1.225 s) : 0, 1224907
Total [candidate] (10.675 s) : 0, 10675343
section iast
Agent [baseline] (1.209 s) : 0, 1208935
Total [baseline] (10.93 s) : 0, 10929908
Agent [candidate] (1.215 s) : 0, 1215381
Total [candidate] (10.896 s) : 0, 10896055
section profiling
Agent [baseline] (1.278 s) : 0, 1278420
Total [baseline] (10.733 s) : 0, 10732524
Agent [candidate] (1.279 s) : 0, 1279127
Total [candidate] (10.707 s) : 0, 10707398
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.081 s -
Agent appsec 1.216 s 134.609 ms (12.5%)
Agent iast 1.209 s 128.035 ms (11.8%)
Agent profiling 1.278 s 197.52 ms (18.3%)
Total tracing 10.399 s -
Total appsec 10.677 s 277.886 ms (2.7%)
Total iast 10.93 s 531.14 ms (5.1%)
Total profiling 10.733 s 333.756 ms (3.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.085 s -
Agent appsec 1.225 s 139.846 ms (12.9%)
Agent iast 1.215 s 130.32 ms (12.0%)
Agent profiling 1.279 s 194.066 ms (17.9%)
Total tracing 10.381 s -
Total appsec 10.675 s 294.226 ms (2.8%)
Total iast 10.896 s 514.938 ms (5.0%)
Total profiling 10.707 s 326.28 ms (3.1%) 
gantt
    title petclinic - break down per module: candidate=1.43.0-SNAPSHOT~7d2a924120, baseline=1.43.0-SNAPSHOT~31bfaf7217

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (687.706 ms) : 0, 687706
BytebuddyAgent [candidate] (688.174 ms) : 0, 688174
GlobalTracer [baseline] (315.986 ms) : 0, 315986
GlobalTracer [candidate] (316.696 ms) : 0, 316696
AppSec [baseline] (54.182 ms) : 0, 54182
AppSec [candidate] (54.292 ms) : 0, 54292
Remote Config [baseline] (676.331 µs) : 0, 676
Remote Config [candidate] (686.168 µs) : 0, 686
Telemetry [baseline] (8.577 ms) : 0, 8577
Telemetry [candidate] (11.433 ms) : 0, 11433
section appsec
BytebuddyAgent [baseline] (703.647 ms) : 0, 703647
BytebuddyAgent [candidate] (710.011 ms) : 0, 710011
GlobalTracer [baseline] (313.096 ms) : 0, 313096
GlobalTracer [candidate] (315.68 ms) : 0, 315680
AppSec [baseline] (166.437 ms) : 0, 166437
AppSec [candidate] (166.86 ms) : 0, 166860
IAST [baseline] (19.68 ms) : 0, 19680
IAST [candidate] (20.232 ms) : 0, 20232
Remote Config [baseline] (643.519 µs) : 0, 644
Remote Config [candidate] (635.715 µs) : 0, 636
Telemetry [baseline] (8.131 ms) : 0, 8131
Telemetry [candidate] (7.781 ms) : 0, 7781
section iast
BytebuddyAgent [baseline] (802.108 ms) : 0, 802108
BytebuddyAgent [candidate] (808.18 ms) : 0, 808180
GlobalTracer [baseline] (306.155 ms) : 0, 306155
GlobalTracer [candidate] (306.404 ms) : 0, 306404
AppSec [baseline] (57.772 ms) : 0, 57772
AppSec [candidate] (56.138 ms) : 0, 56138
IAST [baseline] (20.993 ms) : 0, 20993
IAST [candidate] (22.645 ms) : 0, 22645
Remote Config [baseline] (632.092 µs) : 0, 632
Remote Config [candidate] (627.684 µs) : 0, 628
Telemetry [baseline] (7.544 ms) : 0, 7544
Telemetry [candidate] (7.541 ms) : 0, 7541
section profiling
BytebuddyAgent [baseline] (680.471 ms) : 0, 680471
BytebuddyAgent [candidate] (680.84 ms) : 0, 680840
GlobalTracer [baseline] (399.912 ms) : 0, 399912
GlobalTracer [candidate] (399.507 ms) : 0, 399507
AppSec [baseline] (54.518 ms) : 0, 54518
AppSec [candidate] (54.564 ms) : 0, 54564
Remote Config [baseline] (663.034 µs) : 0, 663
Remote Config [candidate] (663.459 µs) : 0, 663
Telemetry [baseline] (12.014 ms) : 0, 12014
Telemetry [candidate] (12.828 ms) : 0, 12828
ProfilingAgent [baseline] (91.843 ms) : 0, 91843
ProfilingAgent [candidate] (91.821 ms) : 0, 91821
Profiling [baseline] (91.866 ms) : 0, 91866
Profiling [candidate] (91.844 ms) : 0, 91844
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-11-11T17:55:21 2024-11-11T18:04:28
git_branch master smola/spring-boot-3.3-smoke-tests
git_commit_date 1731340376 1731347005
git_commit_sha 31bfaf7 7d2a924
release_version 1.43.0-SNAPSHOT~31bfaf7217 1.43.0-SNAPSHOT~7d2a924120
start_time 2024-11-11T17:55:05 2024-11-11T18:04:11
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1731348731 1731348731
ci_job_id 702530630 702530630
ci_pipeline_id 48724297 48724297
cpu_model Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 7 metrics, 21 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.43.0-SNAPSHOT~7d2a924120, baseline=1.43.0-SNAPSHOT~31bfaf7217
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.704 ms) : 1680, 1729
.   : milestone, 1704,
appsec (2.187 ms) : 2157, 2218
.   : milestone, 2187,
appsec_no_iast (2.203 ms) : 2172, 2234
.   : milestone, 2203,
iast (1.865 ms) : 1835, 1894
.   : milestone, 1865,
profiling (1.903 ms) : 1873, 1933
.   : milestone, 1903,
tracing (1.878 ms) : 1847, 1909
.   : milestone, 1878,
section candidate
no_agent (1.702 ms) : 1678, 1727
.   : milestone, 1702,
appsec (2.214 ms) : 2184, 2244
.   : milestone, 2214,
appsec_no_iast (2.179 ms) : 2147, 2210
.   : milestone, 2179,
iast (1.893 ms) : 1864, 1923
.   : milestone, 1893,
profiling (1.922 ms) : 1888, 1955
.   : milestone, 1922,
tracing (1.881 ms) : 1849, 1912
.   : milestone, 1881,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.704 ms [1.68 ms, 1.729 ms] -
appsec 2.187 ms [2.157 ms, 2.218 ms] 482.845 µs (28.3%)
appsec_no_iast 2.203 ms [2.172 ms, 2.234 ms] 498.829 µs (29.3%)
iast 1.865 ms [1.835 ms, 1.894 ms] 160.208 µs (9.4%)
profiling 1.903 ms [1.873 ms, 1.933 ms] 198.717 µs (11.7%)
tracing 1.878 ms [1.847 ms, 1.909 ms] 173.325 µs (10.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.702 ms [1.678 ms, 1.727 ms] -
appsec 2.214 ms [2.184 ms, 2.244 ms] 511.186 µs (30.0%)
appsec_no_iast 2.179 ms [2.147 ms, 2.21 ms] 476.268 µs (28.0%)
iast 1.893 ms [1.864 ms, 1.923 ms] 190.9 µs (11.2%)
profiling 1.922 ms [1.888 ms, 1.955 ms] 219.227 µs (12.9%)
tracing 1.881 ms [1.849 ms, 1.912 ms] 178.342 µs (10.5%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.43.0-SNAPSHOT~7d2a924120, baseline=1.43.0-SNAPSHOT~31bfaf7217
    dateFormat X
    axisFormat %s
section baseline
no_agent (451.329 µs) : 422, 480
.   : milestone, 451,
iast (584.557 µs) : 553, 616
.   : milestone, 585,
iast_FULL (833.458 µs) : 800, 867
.   : milestone, 833,
iast_GLOBAL (616.338 µs) : 585, 648
.   : milestone, 616,
iast_HARDCODED_SECRET_DISABLED (587.915 µs) : 556, 620
.   : milestone, 588,
iast_INACTIVE (542.953 µs) : 512, 574
.   : milestone, 543,
iast_TELEMETRY_OFF (576.642 µs) : 544, 610
.   : milestone, 577,
tracing (538.729 µs) : 509, 569
.   : milestone, 539,
section candidate
no_agent (441.591 µs) : 413, 470
.   : milestone, 442,
iast (580.687 µs) : 548, 613
.   : milestone, 581,
iast_FULL (838.559 µs) : 805, 872
.   : milestone, 839,
iast_GLOBAL (624.265 µs) : 591, 657
.   : milestone, 624,
iast_HARDCODED_SECRET_DISABLED (586.701 µs) : 555, 619
.   : milestone, 587,
iast_INACTIVE (544.054 µs) : 514, 574
.   : milestone, 544,
iast_TELEMETRY_OFF (577.745 µs) : 545, 610
.   : milestone, 578,
tracing (535.578 µs) : 503, 568
.   : milestone, 536,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 451.329 µs [422.345 µs, 480.314 µs] -
iast 584.557 µs [553.196 µs, 615.918 µs] 133.228 µs (29.5%)
iast_FULL 833.458 µs [800.323 µs, 866.593 µs] 382.129 µs (84.7%)
iast_GLOBAL 616.338 µs [585.036 µs, 647.64 µs] 165.009 µs (36.6%)
iast_HARDCODED_SECRET_DISABLED 587.915 µs [555.798 µs, 620.031 µs] 136.585 µs (30.3%)
iast_INACTIVE 542.953 µs [512.335 µs, 573.571 µs] 91.624 µs (20.3%)
iast_TELEMETRY_OFF 576.642 µs [543.639 µs, 609.645 µs] 125.313 µs (27.8%)
tracing 538.729 µs [508.596 µs, 568.862 µs] 87.4 µs (19.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 441.591 µs [412.789 µs, 470.393 µs] -
iast 580.687 µs [548.132 µs, 613.242 µs] 139.096 µs (31.5%)
iast_FULL 838.559 µs [805.254 µs, 871.865 µs] 396.969 µs (89.9%)
iast_GLOBAL 624.265 µs [591.163 µs, 657.368 µs] 182.674 µs (41.4%)
iast_HARDCODED_SECRET_DISABLED 586.701 µs [554.7 µs, 618.701 µs] 145.11 µs (32.9%)
iast_INACTIVE 544.054 µs [513.901 µs, 574.207 µs] 102.463 µs (23.2%)
iast_TELEMETRY_OFF 577.745 µs [545.11 µs, 610.38 µs] 136.154 µs (30.8%)
tracing 535.578 µs [503.4 µs, 567.756 µs] 93.987 µs (21.3%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/spring-boot-3.3-smoke-tests
git_commit_date 1731340376 1731347005
git_commit_sha 31bfaf7 7d2a924
release_version 1.43.0-SNAPSHOT~31bfaf7217 1.43.0-SNAPSHOT~7d2a924120
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1731348928 1731348928
ci_job_id 702530631 702530631
ci_pipeline_id 48724297 48724297
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.43.0-SNAPSHOT~7d2a924120, baseline=1.43.0-SNAPSHOT~31bfaf7217
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.081 s) : 15081000, 15081000
.   : milestone, 15081000,
appsec (15.334 s) : 15334000, 15334000
.   : milestone, 15334000,
iast (18.952 s) : 18952000, 18952000
.   : milestone, 18952000,
iast_GLOBAL (18.498 s) : 18498000, 18498000
.   : milestone, 18498000,
profiling (15.341 s) : 15341000, 15341000
.   : milestone, 15341000,
tracing (14.964 s) : 14964000, 14964000
.   : milestone, 14964000,
section candidate
no_agent (15.27 s) : 15270000, 15270000
.   : milestone, 15270000,
appsec (15.311 s) : 15311000, 15311000
.   : milestone, 15311000,
iast (18.767 s) : 18767000, 18767000
.   : milestone, 18767000,
iast_GLOBAL (18.338 s) : 18338000, 18338000
.   : milestone, 18338000,
profiling (15.225 s) : 15225000, 15225000
.   : milestone, 15225000,
tracing (15.313 s) : 15313000, 15313000
.   : milestone, 15313000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.081 s [15.081 s, 15.081 s] -
appsec 15.334 s [15.334 s, 15.334 s] 253.0 ms (1.7%)
iast 18.952 s [18.952 s, 18.952 s] 3.871 s (25.7%)
iast_GLOBAL 18.498 s [18.498 s, 18.498 s] 3.417 s (22.7%)
profiling 15.341 s [15.341 s, 15.341 s] 260.0 ms (1.7%)
tracing 14.964 s [14.964 s, 14.964 s] -117.0 ms (-0.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.27 s [15.27 s, 15.27 s] -
appsec 15.311 s [15.311 s, 15.311 s] 41.0 ms (0.3%)
iast 18.767 s [18.767 s, 18.767 s] 3.497 s (22.9%)
iast_GLOBAL 18.338 s [18.338 s, 18.338 s] 3.068 s (20.1%)
profiling 15.225 s [15.225 s, 15.225 s] -45.0 ms (-0.3%)
tracing 15.313 s [15.313 s, 15.313 s] 43.0 ms (0.3%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.43.0-SNAPSHOT~7d2a924120, baseline=1.43.0-SNAPSHOT~31bfaf7217
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.47 ms) : 1459, 1482
.   : milestone, 1470,
appsec (2.31 ms) : 2270, 2350
.   : milestone, 2310,
iast (2.061 ms) : 2010, 2113
.   : milestone, 2061,
iast_GLOBAL (2.116 ms) : 2064, 2169
.   : milestone, 2116,
profiling (1.939 ms) : 1896, 1982
.   : milestone, 1939,
tracing (1.91 ms) : 1871, 1950
.   : milestone, 1910,
section candidate
no_agent (1.461 ms) : 1449, 1472
.   : milestone, 1461,
appsec (2.325 ms) : 2284, 2366
.   : milestone, 2325,
iast (2.062 ms) : 2010, 2114
.   : milestone, 2062,
iast_GLOBAL (2.115 ms) : 2063, 2168
.   : milestone, 2115,
profiling (1.924 ms) : 1883, 1965
.   : milestone, 1924,
tracing (1.914 ms) : 1874, 1954
.   : milestone, 1914,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.47 ms [1.459 ms, 1.482 ms] -
appsec 2.31 ms [2.27 ms, 2.35 ms] 839.764 µs (57.1%)
iast 2.061 ms [2.01 ms, 2.113 ms] 591.209 µs (40.2%)
iast_GLOBAL 2.116 ms [2.064 ms, 2.169 ms] 646.079 µs (43.9%)
profiling 1.939 ms [1.896 ms, 1.982 ms] 468.786 µs (31.9%)
tracing 1.91 ms [1.871 ms, 1.95 ms] 439.983 µs (29.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.461 ms [1.449 ms, 1.472 ms] -
appsec 2.325 ms [2.284 ms, 2.366 ms] 864.245 µs (59.2%)
iast 2.062 ms [2.01 ms, 2.114 ms] 601.339 µs (41.2%)
iast_GLOBAL 2.115 ms [2.063 ms, 2.168 ms] 654.147 µs (44.8%)
profiling 1.924 ms [1.883 ms, 1.965 ms] 463.038 µs (31.7%)
tracing 1.914 ms [1.874 ms, 1.954 ms] 453.115 µs (31.0%)

amarziali
amarziali approved these changes Nov 12, 2024
View reviewed changes
Copy link
Collaborator

@amarziali amarziali left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you

PerfectSlayer
PerfectSlayer approved these changes Nov 12, 2024
View reviewed changes
Copy link
Contributor

@PerfectSlayer PerfectSlayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
I push for investigating if smoke tests could not behave like integration tests, meaning having an unbound max version to test against. WDYT?

@smola
Copy link
Member Author

smola commented Nov 12, 2024

@PerfectSlayer I push for investigating if smoke tests could not behave like integration tests, meaning having an unbound max version to test against. WDYT?

That makes sense to me. It crossed my mind when doing this, since what we'd probably want here is a smoke test for latest Spring Boot 3.x.

@smola smola merged commit 62b2aa0 into master Nov 12, 2024
101 checks passed
@smola smola deleted the smola/spring-boot-3.3-smoke-tests branch November 12, 2024 12:37
@github-actions github-actions bot added this to the 1.43.0 milestone Nov 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

@amarziali amarziali amarziali approved these changes

@dougqh dougqh Awaiting requested review from dougqh dougqh is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: testing Testing tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.43.0
Development

Successfully merging this pull request may close these issues.
3 participants
@smola @PerfectSlayer @amarziali