Skip to content

Adjust CircleCI resource classes #7494

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Sep 17, 2024
Merged

Adjust CircleCI resource classes #7494

merged 5 commits into from
Sep 17, 2024

Conversation

smola
Copy link
Member

@smola smola commented Aug 23, 2024 •
edited
Loading

What Does This Do

See Circle CI's price list. Linux VM / (x86) Remote Docker's xlarge has 8 CPUs for 100 credits/min (12.5 credits per CPU/min), large has 4 CPUs for 20 credits/min (5 credits per CPU/min). So:

Additional Notes

Next in the pipeline: test tasks that gradle does not parallelize within the same subproject now dominate wall clock time in instrumentation test jobs. This is supposed to be eventually fixed by enabling Gradle's configuration cache.

Contributor Checklist

@pr-commenter
Copy link

pr-commenter bot commented Aug 23, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/optimize-resource-classes-circleci
git_commit_date 1726496632 1726501630
git_commit_sha c6e9d40 8214b51
release_version 1.40.0-SNAPSHOT~c6e9d40308 1.40.0-SNAPSHOT~8214b5126b
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1726504112 1726504112
ci_job_id 640835247 640835247
ci_pipeline_id 44429080 44429080
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 17 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.40.0-SNAPSHOT~8214b5126b, baseline=1.40.0-SNAPSHOT~c6e9d40308

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1062510
Total [baseline] (10.277 s) : 0, 10277324
Agent [candidate] (1.064 s) : 0, 1063901
Total [candidate] (10.34 s) : 0, 10339620
section appsec
Agent [baseline] (1.202 s) : 0, 1202019
Total [baseline] (10.679 s) : 0, 10678944
Agent [candidate] (1.199 s) : 0, 1199162
Total [candidate] (10.616 s) : 0, 10616408
section iast
Agent [baseline] (1.202 s) : 0, 1202459
Total [baseline] (10.87 s) : 0, 10869650
Agent [candidate] (1.203 s) : 0, 1203355
Total [candidate] (10.911 s) : 0, 10911408
section profiling
Agent [baseline] (1.259 s) : 0, 1258851
Total [baseline] (10.557 s) : 0, 10557380
Agent [candidate] (1.264 s) : 0, 1263893
Total [candidate] (10.607 s) : 0, 10607103
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent appsec 1.202 s 139.509 ms (13.1%)
Agent iast 1.202 s 139.949 ms (13.2%)
Agent profiling 1.259 s 196.341 ms (18.5%)
Total tracing 10.277 s -
Total appsec 10.679 s 401.621 ms (3.9%)
Total iast 10.87 s 592.326 ms (5.8%)
Total profiling 10.557 s 280.056 ms (2.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent appsec 1.199 s 135.261 ms (12.7%)
Agent iast 1.203 s 139.454 ms (13.1%)
Agent profiling 1.264 s 199.992 ms (18.8%)
Total tracing 10.34 s -
Total appsec 10.616 s 276.789 ms (2.7%)
Total iast 10.911 s 571.788 ms (5.5%)
Total profiling 10.607 s 267.483 ms (2.6%) 
gantt
    title petclinic - break down per module: candidate=1.40.0-SNAPSHOT~8214b5126b, baseline=1.40.0-SNAPSHOT~c6e9d40308

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (677.948 ms) : 0, 677948
BytebuddyAgent [candidate] (678.464 ms) : 0, 678464
GlobalTracer [baseline] (311.092 ms) : 0, 311092
GlobalTracer [candidate] (311.714 ms) : 0, 311714
AppSec [baseline] (51.747 ms) : 0, 51747
AppSec [candidate] (51.976 ms) : 0, 51976
Remote Config [baseline] (665.425 µs) : 0, 665
Remote Config [candidate] (659.497 µs) : 0, 659
Telemetry [baseline] (7.457 ms) : 0, 7457
Telemetry [candidate] (7.471 ms) : 0, 7471
section appsec
BytebuddyAgent [baseline] (702.684 ms) : 0, 702684
BytebuddyAgent [candidate] (701.725 ms) : 0, 701725
GlobalTracer [baseline] (305.325 ms) : 0, 305325
GlobalTracer [candidate] (303.824 ms) : 0, 303824
AppSec [baseline] (160.847 ms) : 0, 160847
AppSec [candidate] (159.081 ms) : 0, 159081
IAST [baseline] (20.921 ms) : 0, 20921
IAST [candidate] (21.6 ms) : 0, 21600
Remote Config [baseline] (655.313 µs) : 0, 655
Remote Config [candidate] (1.015 ms) : 0, 1015
Telemetry [baseline] (8.426 ms) : 0, 8426
Telemetry [candidate] (9.084 ms) : 0, 9084
section iast
BytebuddyAgent [baseline] (801.191 ms) : 0, 801191
BytebuddyAgent [candidate] (800.888 ms) : 0, 800888
GlobalTracer [baseline] (301.959 ms) : 0, 301959
GlobalTracer [candidate] (302.982 ms) : 0, 302982
AppSec [baseline] (54.557 ms) : 0, 54557
AppSec [candidate] (54.455 ms) : 0, 54455
IAST [baseline] (22.916 ms) : 0, 22916
IAST [candidate] (23.104 ms) : 0, 23104
Remote Config [baseline] (634.265 µs) : 0, 634
Remote Config [candidate] (638.407 µs) : 0, 638
Telemetry [baseline] (7.393 ms) : 0, 7393
Telemetry [candidate] (7.475 ms) : 0, 7475
section profiling
BytebuddyAgent [baseline] (671.388 ms) : 0, 671388
BytebuddyAgent [candidate] (673.516 ms) : 0, 673516
GlobalTracer [baseline] (392.684 ms) : 0, 392684
GlobalTracer [candidate] (394.609 ms) : 0, 394609
AppSec [baseline] (52.615 ms) : 0, 52615
AppSec [candidate] (53.056 ms) : 0, 53056
Remote Config [baseline] (663.504 µs) : 0, 664
Remote Config [candidate] (670.114 µs) : 0, 670
Telemetry [baseline] (7.366 ms) : 0, 7366
Telemetry [candidate] (7.475 ms) : 0, 7475
ProfilingAgent [baseline] (96.253 ms) : 0, 96253
ProfilingAgent [candidate] (96.536 ms) : 0, 96536
Profiling [baseline] (96.277 ms) : 0, 96277
Profiling [candidate] (96.56 ms) : 0, 96560
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.40.0-SNAPSHOT~8214b5126b, baseline=1.40.0-SNAPSHOT~c6e9d40308

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.077 s) : 0, 1077436
Total [baseline] (8.537 s) : 0, 8536934
Agent [candidate] (1.073 s) : 0, 1072806
Total [candidate] (8.541 s) : 0, 8541324
section iast
Agent [baseline] (1.19 s) : 0, 1189762
Total [baseline] (8.989 s) : 0, 8989193
Agent [candidate] (1.191 s) : 0, 1191258
Total [candidate] (9.001 s) : 0, 9001126
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.186 s) : 0, 1185990
Total [baseline] (8.972 s) : 0, 8971541
Agent [candidate] (1.19 s) : 0, 1190337
Total [candidate] (8.948 s) : 0, 8948068
section iast_TELEMETRY_OFF
Agent [baseline] (1.185 s) : 0, 1185406
Total [baseline] (8.967 s) : 0, 8966989
Agent [candidate] (1.194 s) : 0, 1194072
Total [candidate] (8.98 s) : 0, 8980440
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.077 s -
Agent iast 1.19 s 112.326 ms (10.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.186 s 108.554 ms (10.1%)
Agent iast_TELEMETRY_OFF 1.185 s 107.969 ms (10.0%)
Total tracing 8.537 s -
Total iast 8.989 s 452.259 ms (5.3%)
Total iast_HARDCODED_SECRET_DISABLED 8.972 s 434.607 ms (5.1%)
Total iast_TELEMETRY_OFF 8.967 s 430.055 ms (5.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.073 s -
Agent iast 1.191 s 118.452 ms (11.0%)
Agent iast_HARDCODED_SECRET_DISABLED 1.19 s 117.531 ms (11.0%)
Agent iast_TELEMETRY_OFF 1.194 s 121.266 ms (11.3%)
Total tracing 8.541 s -
Total iast 9.001 s 459.802 ms (5.4%)
Total iast_HARDCODED_SECRET_DISABLED 8.948 s 406.744 ms (4.8%)
Total iast_TELEMETRY_OFF 8.98 s 439.116 ms (5.1%) 
gantt
    title insecure-bank - break down per module: candidate=1.40.0-SNAPSHOT~8214b5126b, baseline=1.40.0-SNAPSHOT~c6e9d40308

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (689.286 ms) : 0, 689286
BytebuddyAgent [candidate] (684.701 ms) : 0, 684701
GlobalTracer [baseline] (313.504 ms) : 0, 313504
GlobalTracer [candidate] (313.809 ms) : 0, 313809
AppSec [baseline] (52.535 ms) : 0, 52535
AppSec [candidate] (52.367 ms) : 0, 52367
Remote Config [baseline] (681.739 µs) : 0, 682
Remote Config [candidate] (671.959 µs) : 0, 672
Telemetry [baseline] (7.619 ms) : 0, 7619
Telemetry [candidate] (7.515 ms) : 0, 7515
section iast
BytebuddyAgent [baseline] (792.496 ms) : 0, 792496
BytebuddyAgent [candidate] (792.852 ms) : 0, 792852
GlobalTracer [baseline] (299.156 ms) : 0, 299156
GlobalTracer [candidate] (300.049 ms) : 0, 300049
AppSec [baseline] (52.849 ms) : 0, 52849
AppSec [candidate] (52.952 ms) : 0, 52952
IAST [baseline] (23.649 ms) : 0, 23649
IAST [candidate] (22.847 ms) : 0, 22847
Remote Config [baseline] (602.54 µs) : 0, 603
Remote Config [candidate] (647.004 µs) : 0, 647
Telemetry [baseline] (7.312 ms) : 0, 7312
Telemetry [candidate] (8.216 ms) : 0, 8216
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (789.588 ms) : 0, 789588
BytebuddyAgent [candidate] (792.444 ms) : 0, 792444
GlobalTracer [baseline] (298.042 ms) : 0, 298042
GlobalTracer [candidate] (299.504 ms) : 0, 299504
AppSec [baseline] (51.816 ms) : 0, 51816
AppSec [candidate] (50.778 ms) : 0, 50778
IAST [baseline] (25.036 ms) : 0, 25036
IAST [candidate] (25.154 ms) : 0, 25154
Remote Config [baseline] (624.51 µs) : 0, 625
Remote Config [candidate] (668.368 µs) : 0, 668
Telemetry [baseline] (7.269 ms) : 0, 7269
Telemetry [candidate] (8.096 ms) : 0, 8096
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (787.822 ms) : 0, 787822
BytebuddyAgent [candidate] (793.678 ms) : 0, 793678
GlobalTracer [baseline] (299.497 ms) : 0, 299497
GlobalTracer [candidate] (301.614 ms) : 0, 301614
AppSec [baseline] (50.988 ms) : 0, 50988
AppSec [candidate] (52.085 ms) : 0, 52085
IAST [baseline] (24.848 ms) : 0, 24848
IAST [candidate] (24.324 ms) : 0, 24324
Remote Config [baseline] (603.356 µs) : 0, 603
Remote Config [candidate] (608.991 µs) : 0, 609
Telemetry [baseline] (8.014 ms) : 0, 8014
Telemetry [candidate] (8.012 ms) : 0, 8012
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-09-16T15:59:15 2024-09-16T16:06:05
git_branch master smola/optimize-resource-classes-circleci
git_commit_date 1726496632 1726501630
git_commit_sha c6e9d40 8214b51
release_version 1.40.0-SNAPSHOT~c6e9d40308 1.40.0-SNAPSHOT~8214b5126b
start_time 2024-09-16T15:59:02 2024-09-16T16:05:51
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1726503111 1726503111
ci_job_id 640835248 640835248
ci_pipeline_id 44429080 44429080
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~8214b5126b, baseline=1.40.0-SNAPSHOT~c6e9d40308
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.342 ms) : 1323, 1361
.   : milestone, 1342,
appsec (1.705 ms) : 1681, 1730
.   : milestone, 1705,
appsec_no_iast (1.723 ms) : 1700, 1747
.   : milestone, 1723,
iast (1.476 ms) : 1454, 1499
.   : milestone, 1476,
profiling (1.47 ms) : 1447, 1493
.   : milestone, 1470,
tracing (1.477 ms) : 1453, 1501
.   : milestone, 1477,
section candidate
no_agent (1.341 ms) : 1322, 1360
.   : milestone, 1341,
appsec (1.735 ms) : 1711, 1759
.   : milestone, 1735,
appsec_no_iast (1.727 ms) : 1702, 1752
.   : milestone, 1727,
iast (1.476 ms) : 1453, 1498
.   : milestone, 1476,
profiling (1.489 ms) : 1466, 1512
.   : milestone, 1489,
tracing (1.455 ms) : 1430, 1479
.   : milestone, 1455,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.342 ms [1.323 ms, 1.361 ms] -
appsec 1.705 ms [1.681 ms, 1.73 ms] 363.336 µs (27.1%)
appsec_no_iast 1.723 ms [1.7 ms, 1.747 ms] 381.342 µs (28.4%)
iast 1.476 ms [1.454 ms, 1.499 ms] 134.097 µs (10.0%)
profiling 1.47 ms [1.447 ms, 1.493 ms] 127.413 µs (9.5%)
tracing 1.477 ms [1.453 ms, 1.501 ms] 134.953 µs (10.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.341 ms [1.322 ms, 1.36 ms] -
appsec 1.735 ms [1.711 ms, 1.759 ms] 393.625 µs (29.4%)
appsec_no_iast 1.727 ms [1.702 ms, 1.752 ms] 385.865 µs (28.8%)
iast 1.476 ms [1.453 ms, 1.498 ms] 134.62 µs (10.0%)
profiling 1.489 ms [1.466 ms, 1.512 ms] 148.001 µs (11.0%)
tracing 1.455 ms [1.43 ms, 1.479 ms] 113.757 µs (8.5%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~8214b5126b, baseline=1.40.0-SNAPSHOT~c6e9d40308
    dateFormat X
    axisFormat %s
section baseline
no_agent (373.896 µs) : 354, 394
.   : milestone, 374,
iast (485.938 µs) : 464, 507
.   : milestone, 486,
iast_FULL (552.607 µs) : 531, 574
.   : milestone, 553,
iast_GLOBAL (506.483 µs) : 484, 529
.   : milestone, 506,
iast_HARDCODED_SECRET_DISABLED (482.047 µs) : 461, 503
.   : milestone, 482,
iast_INACTIVE (448.487 µs) : 428, 469
.   : milestone, 448,
iast_TELEMETRY_OFF (472.72 µs) : 450, 495
.   : milestone, 473,
tracing (439.48 µs) : 419, 460
.   : milestone, 439,
section candidate
no_agent (371.286 µs) : 352, 391
.   : milestone, 371,
iast (483.264 µs) : 462, 505
.   : milestone, 483,
iast_FULL (552.839 µs) : 532, 574
.   : milestone, 553,
iast_GLOBAL (514.975 µs) : 493, 537
.   : milestone, 515,
iast_HARDCODED_SECRET_DISABLED (480.594 µs) : 460, 502
.   : milestone, 481,
iast_INACTIVE (446.574 µs) : 425, 468
.   : milestone, 447,
iast_TELEMETRY_OFF (480.506 µs) : 458, 503
.   : milestone, 481,
tracing (443.031 µs) : 423, 464
.   : milestone, 443,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 373.896 µs [353.759 µs, 394.033 µs] -
iast 485.938 µs [464.422 µs, 507.453 µs] 112.042 µs (30.0%)
iast_FULL 552.607 µs [531.389 µs, 573.825 µs] 178.71 µs (47.8%)
iast_GLOBAL 506.483 µs [484.257 µs, 528.709 µs] 132.587 µs (35.5%)
iast_HARDCODED_SECRET_DISABLED 482.047 µs [461.011 µs, 503.083 µs] 108.15 µs (28.9%)
iast_INACTIVE 448.487 µs [427.54 µs, 469.435 µs] 74.591 µs (19.9%)
iast_TELEMETRY_OFF 472.72 µs [450.326 µs, 495.115 µs] 98.824 µs (26.4%)
tracing 439.48 µs [419.406 µs, 459.555 µs] 65.584 µs (17.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 371.286 µs [351.757 µs, 390.815 µs] -
iast 483.264 µs [461.649 µs, 504.879 µs] 111.978 µs (30.2%)
iast_FULL 552.839 µs [531.697 µs, 573.981 µs] 181.553 µs (48.9%)
iast_GLOBAL 514.975 µs [492.551 µs, 537.399 µs] 143.689 µs (38.7%)
iast_HARDCODED_SECRET_DISABLED 480.594 µs [459.607 µs, 501.58 µs] 109.308 µs (29.4%)
iast_INACTIVE 446.574 µs [425.363 µs, 467.785 µs] 75.288 µs (20.3%)
iast_TELEMETRY_OFF 480.506 µs [457.799 µs, 503.213 µs] 109.22 µs (29.4%)
tracing 443.031 µs [422.555 µs, 463.507 µs] 71.745 µs (19.3%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/optimize-resource-classes-circleci
git_commit_date 1726496632 1726501630
git_commit_sha c6e9d40 8214b51
release_version 1.40.0-SNAPSHOT~c6e9d40308 1.40.0-SNAPSHOT~8214b5126b
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1726503677 1726503677
ci_job_id 640835249 640835249
ci_pipeline_id 44429080 44429080
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~8214b5126b, baseline=1.40.0-SNAPSHOT~c6e9d40308
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.807 s) : 15807000, 15807000
.   : milestone, 15807000,
appsec (15.187 s) : 15187000, 15187000
.   : milestone, 15187000,
iast (18.814 s) : 18814000, 18814000
.   : milestone, 18814000,
iast_GLOBAL (18.143 s) : 18143000, 18143000
.   : milestone, 18143000,
profiling (15.308 s) : 15308000, 15308000
.   : milestone, 15308000,
tracing (15.067 s) : 15067000, 15067000
.   : milestone, 15067000,
section candidate
no_agent (15.388 s) : 15388000, 15388000
.   : milestone, 15388000,
appsec (15.12 s) : 15120000, 15120000
.   : milestone, 15120000,
iast (19.099 s) : 19099000, 19099000
.   : milestone, 19099000,
iast_GLOBAL (18.039 s) : 18039000, 18039000
.   : milestone, 18039000,
profiling (15.219 s) : 15219000, 15219000
.   : milestone, 15219000,
tracing (14.96 s) : 14960000, 14960000
.   : milestone, 14960000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.807 s [15.807 s, 15.807 s] -
appsec 15.187 s [15.187 s, 15.187 s] -620.0 ms (-3.9%)
iast 18.814 s [18.814 s, 18.814 s] 3.007 s (19.0%)
iast_GLOBAL 18.143 s [18.143 s, 18.143 s] 2.336 s (14.8%)
profiling 15.308 s [15.308 s, 15.308 s] -499.0 ms (-3.2%)
tracing 15.067 s [15.067 s, 15.067 s] -740.0 ms (-4.7%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.388 s [15.388 s, 15.388 s] -
appsec 15.12 s [15.12 s, 15.12 s] -268.0 ms (-1.7%)
iast 19.099 s [19.099 s, 19.099 s] 3.711 s (24.1%)
iast_GLOBAL 18.039 s [18.039 s, 18.039 s] 2.651 s (17.2%)
profiling 15.219 s [15.219 s, 15.219 s] -169.0 ms (-1.1%)
tracing 14.96 s [14.96 s, 14.96 s] -428.0 ms (-2.8%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~8214b5126b, baseline=1.40.0-SNAPSHOT~c6e9d40308
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.463 ms) : 1452, 1475
.   : milestone, 1463,
appsec (2.292 ms) : 2251, 2332
.   : milestone, 2292,
iast (2.052 ms) : 2001, 2103
.   : milestone, 2052,
iast_GLOBAL (2.09 ms) : 2039, 2141
.   : milestone, 2090,
profiling (1.929 ms) : 1888, 1970
.   : milestone, 1929,
tracing (1.893 ms) : 1856, 1931
.   : milestone, 1893,
section candidate
no_agent (1.462 ms) : 1451, 1474
.   : milestone, 1462,
appsec (2.3 ms) : 2259, 2341
.   : milestone, 2300,
iast (2.061 ms) : 2010, 2111
.   : milestone, 2061,
iast_GLOBAL (2.096 ms) : 2045, 2147
.   : milestone, 2096,
profiling (1.928 ms) : 1886, 1969
.   : milestone, 1928,
tracing (1.897 ms) : 1859, 1935
.   : milestone, 1897,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.463 ms [1.452 ms, 1.475 ms] -
appsec 2.292 ms [2.251 ms, 2.332 ms] 828.265 µs (56.6%)
iast 2.052 ms [2.001 ms, 2.103 ms] 588.799 µs (40.2%)
iast_GLOBAL 2.09 ms [2.039 ms, 2.141 ms] 626.588 µs (42.8%)
profiling 1.929 ms [1.888 ms, 1.97 ms] 465.287 µs (31.8%)
tracing 1.893 ms [1.856 ms, 1.931 ms] 429.946 µs (29.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.462 ms [1.451 ms, 1.474 ms] -
appsec 2.3 ms [2.259 ms, 2.341 ms] 837.574 µs (57.3%)
iast 2.061 ms [2.01 ms, 2.111 ms] 598.199 µs (40.9%)
iast_GLOBAL 2.096 ms [2.045 ms, 2.147 ms] 633.438 µs (43.3%)
profiling 1.928 ms [1.886 ms, 1.969 ms] 465.114 µs (31.8%)
tracing 1.897 ms [1.859 ms, 1.935 ms] 434.384 µs (29.7%)

@smola smola force-pushed the smola/optimize-resource-classes-circleci branch 12 times, most recently from a6fe02e to a565672 Compare September 4, 2024 10:05
@smola smola force-pushed the smola/optimize-resource-classes-circleci branch 8 times, most recently from e3238e9 to e22f0c3 Compare September 13, 2024 07:36
@smola smola added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Sep 13, 2024
@smola smola force-pushed the smola/optimize-resource-classes-circleci branch 2 times, most recently from 083385f to 03310bc Compare September 16, 2024 09:28
@smola smola marked this pull request as ready for review September 16, 2024 10:48
@smola smola requested review from a team as code owners September 16, 2024 10:48
@smola smola force-pushed the smola/optimize-resource-classes-circleci branch from 03310bc to 4bc8ad7 Compare September 16, 2024 11:20
@smola smola force-pushed the smola/optimize-resource-classes-circleci branch from 4bc8ad7 to ada6c6f Compare September 16, 2024 14:24
PerfectSlayer
PerfectSlayer approved these changes Sep 17, 2024
View reviewed changes
.circleci/config.continue.yml.j2
@@ -660,7 +660,8 @@ jobs:
- DD_POOL_TRACE_CHECK_FAILURES=true
- DD_DISABLE_ERROR_RESPONSES=true
- ENABLED_CHECKS=trace_content_length,trace_stall,meta_tracer_version_header,trace_count_header,trace_peer_service,trace_dd_service
resource_class: xlarge
# TODO: merge xlarge_tests and tests? or rename this?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leftover or notes for later?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note for later. I minimized changes in this PR, but it's a good idea to refactor/rename the tests and xlarge_tests blocks to better describe what they are about (which is not about resource classes at all, but other setup aspects).

@smola smola merged commit 2d2c029 into master Sep 17, 2024
102 checks passed
@smola smola deleted the smola/optimize-resource-classes-circleci branch September 17, 2024 12:45
@github-actions github-actions bot added this to the 1.40.0 milestone Sep 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

@manuel-alvarez-alvarez manuel-alvarez-alvarez Awaiting requested review from manuel-alvarez-alvarez manuel-alvarez-alvarez is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.40.0
Development

Successfully merging this pull request may close these issues.
2 participants
@smola @PerfectSlayer