Update JUnit Tests to run with STRICT_TEST mode by default by mhlidd · Pull Request #11045 · DataDog/dd-trace-java

mhlidd · 2026-04-02T20:50:43Z

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

pr-commenter · 2026-04-03T16:40:35Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mhlidd/junit_strict_mode
git_commit_date	1775065969	1775278168
git_commit_sha	`e307e2c`	`cc76233`
release_version	1.61.0-SNAPSHOT~e307e2c163	1.61.0-SNAPSHOT~cc76233e82

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1775280085	1775280085
ci_job_id	1568259459	1568259459
ci_pipeline_id	106065531	106065531
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-n9gky61y 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-n9gky61y 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 14 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~cc76233e82, baseline=1.61.0-SNAPSHOT~e307e2c163

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.055 s) : 0, 1054848
Total [baseline] (8.85 s) : 0, 8850066
Agent [candidate] (1.06 s) : 0, 1059936
Total [candidate] (8.832 s) : 0, 8832408
section iast
Agent [baseline] (1.225 s) : 0, 1224970
Total [baseline] (9.556 s) : 0, 9555607
Agent [candidate] (1.229 s) : 0, 1228600
Total [candidate] (9.545 s) : 0, 9545449

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.055 s	-
Agent	iast	1.225 s	170.122 ms (16.1%)
Total	tracing	8.85 s	-
Total	iast	9.556 s	705.541 ms (8.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.06 s	-
Agent	iast	1.229 s	168.664 ms (15.9%)
Total	tracing	8.832 s	-
Total	iast	9.545 s	713.041 ms (8.1%)

gantt
    title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~cc76233e82, baseline=1.61.0-SNAPSHOT~e307e2c163

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.194 ms) : 0, 1194
crashtracking [candidate] (1.224 ms) : 0, 1224
BytebuddyAgent [baseline] (633.817 ms) : 0, 633817
BytebuddyAgent [candidate] (635.346 ms) : 0, 635346
AgentMeter [baseline] (29.465 ms) : 0, 29465
AgentMeter [candidate] (29.355 ms) : 0, 29355
GlobalTracer [baseline] (248.666 ms) : 0, 248666
GlobalTracer [candidate] (248.894 ms) : 0, 248894
AppSec [baseline] (31.988 ms) : 0, 31988
AppSec [candidate] (32.081 ms) : 0, 32081
Debugger [baseline] (59.147 ms) : 0, 59147
Debugger [candidate] (59.232 ms) : 0, 59232
Remote Config [baseline] (610.28 µs) : 0, 610
Remote Config [candidate] (594.316 µs) : 0, 594
Telemetry [baseline] (8.037 ms) : 0, 8037
Telemetry [candidate] (8.124 ms) : 0, 8124
Flare Poller [baseline] (5.861 ms) : 0, 5861
Flare Poller [candidate] (8.967 ms) : 0, 8967
section iast
crashtracking [baseline] (1.19 ms) : 0, 1190
crashtracking [candidate] (1.211 ms) : 0, 1211
BytebuddyAgent [baseline] (802.874 ms) : 0, 802874
BytebuddyAgent [candidate] (805.808 ms) : 0, 805808
AgentMeter [baseline] (11.421 ms) : 0, 11421
AgentMeter [candidate] (11.635 ms) : 0, 11635
GlobalTracer [baseline] (239.044 ms) : 0, 239044
GlobalTracer [candidate] (239.311 ms) : 0, 239311
AppSec [baseline] (32.581 ms) : 0, 32581
AppSec [candidate] (31.054 ms) : 0, 31054
Debugger [baseline] (56.585 ms) : 0, 56585
Debugger [candidate] (57.434 ms) : 0, 57434
Remote Config [baseline] (513.907 µs) : 0, 514
Remote Config [candidate] (527.306 µs) : 0, 527
Telemetry [baseline] (14.608 ms) : 0, 14608
Telemetry [candidate] (14.334 ms) : 0, 14334
Flare Poller [baseline] (3.768 ms) : 0, 3768
Flare Poller [candidate] (4.401 ms) : 0, 4401
IAST [baseline] (25.775 ms) : 0, 25775
IAST [candidate] (26.736 ms) : 0, 26736

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~cc76233e82, baseline=1.61.0-SNAPSHOT~e307e2c163

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.055 s) : 0, 1055229
Total [baseline] (11.12 s) : 0, 11119886
Agent [candidate] (1.066 s) : 0, 1066191
Total [candidate] (11.054 s) : 0, 11054257
section appsec
Agent [baseline] (1.248 s) : 0, 1248129
Total [baseline] (11.129 s) : 0, 11128906
Agent [candidate] (1.248 s) : 0, 1248110
Total [candidate] (11.153 s) : 0, 11153294
section iast
Agent [baseline] (1.225 s) : 0, 1225411
Total [baseline] (11.277 s) : 0, 11276796
Agent [candidate] (1.225 s) : 0, 1224885
Total [candidate] (11.313 s) : 0, 11312864
section profiling
Agent [baseline] (1.19 s) : 0, 1189773
Total [baseline] (11.068 s) : 0, 11067564
Agent [candidate] (1.186 s) : 0, 1186310
Total [candidate] (11.123 s) : 0, 11123231

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.055 s	-
Agent	appsec	1.248 s	192.9 ms (18.3%)
Agent	iast	1.225 s	170.183 ms (16.1%)
Agent	profiling	1.19 s	134.545 ms (12.8%)
Total	tracing	11.12 s	-
Total	appsec	11.129 s	9.02 ms (0.1%)
Total	iast	11.277 s	156.91 ms (1.4%)
Total	profiling	11.068 s	-52.322 ms (-0.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.066 s	-
Agent	appsec	1.248 s	181.918 ms (17.1%)
Agent	iast	1.225 s	158.694 ms (14.9%)
Agent	profiling	1.186 s	120.118 ms (11.3%)
Total	tracing	11.054 s	-
Total	appsec	11.153 s	99.037 ms (0.9%)
Total	iast	11.313 s	258.607 ms (2.3%)
Total	profiling	11.123 s	68.974 ms (0.6%)

gantt
    title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~cc76233e82, baseline=1.61.0-SNAPSHOT~e307e2c163

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.216 ms) : 0, 1216
crashtracking [candidate] (1.202 ms) : 0, 1202
BytebuddyAgent [baseline] (633.644 ms) : 0, 633644
BytebuddyAgent [candidate] (638.206 ms) : 0, 638206
AgentMeter [baseline] (29.472 ms) : 0, 29472
AgentMeter [candidate] (29.63 ms) : 0, 29630
GlobalTracer [baseline] (248.027 ms) : 0, 248027
GlobalTracer [candidate] (250.307 ms) : 0, 250307
AppSec [baseline] (31.921 ms) : 0, 31921
AppSec [candidate] (32.091 ms) : 0, 32091
Debugger [baseline] (59.613 ms) : 0, 59613
Debugger [candidate] (60.147 ms) : 0, 60147
Remote Config [baseline] (591.829 µs) : 0, 592
Remote Config [candidate] (596.99 µs) : 0, 597
Telemetry [baseline] (8.056 ms) : 0, 8056
Telemetry [candidate] (8.146 ms) : 0, 8146
Flare Poller [baseline] (6.513 ms) : 0, 6513
Flare Poller [candidate] (9.599 ms) : 0, 9599
section appsec
crashtracking [baseline] (1.193 ms) : 0, 1193
crashtracking [candidate] (1.205 ms) : 0, 1205
BytebuddyAgent [baseline] (662.775 ms) : 0, 662775
BytebuddyAgent [candidate] (663.204 ms) : 0, 663204
AgentMeter [baseline] (12.096 ms) : 0, 12096
AgentMeter [candidate] (12.045 ms) : 0, 12045
GlobalTracer [baseline] (248.688 ms) : 0, 248688
GlobalTracer [candidate] (248.816 ms) : 0, 248816
AppSec [baseline] (183.857 ms) : 0, 183857
AppSec [candidate] (183.099 ms) : 0, 183099
Debugger [baseline] (66.059 ms) : 0, 66059
Debugger [candidate] (66.169 ms) : 0, 66169
Remote Config [baseline] (617.43 µs) : 0, 617
Remote Config [candidate] (609.395 µs) : 0, 609
Telemetry [baseline] (8.656 ms) : 0, 8656
Telemetry [candidate] (8.625 ms) : 0, 8625
Flare Poller [baseline] (3.513 ms) : 0, 3513
Flare Poller [candidate] (3.546 ms) : 0, 3546
IAST [baseline] (24.418 ms) : 0, 24418
IAST [candidate] (24.529 ms) : 0, 24529
section iast
crashtracking [baseline] (1.191 ms) : 0, 1191
crashtracking [candidate] (1.231 ms) : 0, 1231
BytebuddyAgent [baseline] (800.578 ms) : 0, 800578
BytebuddyAgent [candidate] (801.508 ms) : 0, 801508
AgentMeter [baseline] (11.448 ms) : 0, 11448
AgentMeter [candidate] (11.385 ms) : 0, 11385
GlobalTracer [baseline] (239.569 ms) : 0, 239569
GlobalTracer [candidate] (239.189 ms) : 0, 239189
AppSec [baseline] (31.256 ms) : 0, 31256
AppSec [candidate] (29.315 ms) : 0, 29315
Debugger [baseline] (58.848 ms) : 0, 58848
Debugger [candidate] (61.045 ms) : 0, 61045
Remote Config [baseline] (543.376 µs) : 0, 543
Remote Config [candidate] (544.135 µs) : 0, 544
Telemetry [baseline] (14.594 ms) : 0, 14594
Telemetry [candidate] (14.383 ms) : 0, 14383
Flare Poller [baseline] (4.495 ms) : 0, 4495
Flare Poller [candidate] (4.127 ms) : 0, 4127
IAST [baseline] (26.79 ms) : 0, 26790
IAST [candidate] (25.835 ms) : 0, 25835
section profiling
crashtracking [baseline] (1.191 ms) : 0, 1191
crashtracking [candidate] (1.181 ms) : 0, 1181
BytebuddyAgent [baseline] (695.454 ms) : 0, 695454
BytebuddyAgent [candidate] (692.977 ms) : 0, 692977
AgentMeter [baseline] (9.215 ms) : 0, 9215
AgentMeter [candidate] (9.138 ms) : 0, 9138
GlobalTracer [baseline] (207.961 ms) : 0, 207961
GlobalTracer [candidate] (207.098 ms) : 0, 207098
AppSec [baseline] (32.654 ms) : 0, 32654
AppSec [candidate] (32.685 ms) : 0, 32685
Debugger [baseline] (66.222 ms) : 0, 66222
Debugger [candidate] (66.133 ms) : 0, 66133
Remote Config [baseline] (560.019 µs) : 0, 560
Remote Config [candidate] (577.949 µs) : 0, 578
Telemetry [baseline] (7.794 ms) : 0, 7794
Telemetry [candidate] (7.874 ms) : 0, 7874
Flare Poller [baseline] (3.611 ms) : 0, 3611
Flare Poller [candidate] (3.655 ms) : 0, 3655
ProfilingAgent [baseline] (93.58 ms) : 0, 93580
ProfilingAgent [candidate] (93.849 ms) : 0, 93849
Profiling [baseline] (94.15 ms) : 0, 94150
Profiling [candidate] (94.412 ms) : 0, 94412

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mhlidd/junit_strict_mode
git_commit_date	1775065969	1775278168
git_commit_sha	`e307e2c`	`cc76233`
release_version	1.61.0-SNAPSHOT~e307e2c163	1.61.0-SNAPSHOT~cc76233e82

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1775280691	1775280691
ci_job_id	1568259460	1568259460
ci_pipeline_id	106065531	106065531
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-sfpurn3z 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-sfpurn3z 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 2 performance regressions! Performance is the same for 15 metrics, 17 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:insecure-bank:iast_GLOBAL:high_load	better [-230.200µs; -118.804µs] or [-7.725%; -3.987%]	unsure [-489.108µs; -89.848µs] or [-5.973%; -1.097%]	unstable [-68.111op/s; +188.486op/s] or [-5.545%; +15.344%]	2.805ms	7.899ms	1288.594op/s	2.980ms	8.189ms	1228.406op/s
scenario:load:insecure-bank:iast:high_load	better [-263.557µs; -94.589µs] or [-9.801%; -3.518%]	unstable [-1217.955µs; +2009.751µs] or [-15.750%; +25.989%]	unstable [-90.100op/s; +204.100op/s] or [-6.651%; +15.065%]	2.510ms	8.129ms	1411.781op/s	2.689ms	7.733ms	1354.781op/s
scenario:load:petclinic:appsec:high_load	worse [+0.669ms; +1.445ms] or [+3.646%; +7.867%]	unsure [+78.673µs; +1670.453µs] or [+0.262%; +5.562%]	unstable [-36.740op/s; +12.927op/s] or [-14.714%; +5.177%]	19.420ms	30.906ms	237.781op/s	18.363ms	30.032ms	249.688op/s
scenario:load:petclinic:no_agent:high_load	worse [+0.373ms; +2.077ms] or [+2.096%; +11.680%]	unstable [-0.516ms; +3.102ms] or [-1.733%; +10.421%]	unstable [-43.821op/s; +9.821op/s] or [-17.020%; +3.815%]	19.011ms	31.060ms	240.469op/s	17.786ms	29.767ms	257.469op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~cc76233e82, baseline=1.61.0-SNAPSHOT~e307e2c163
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.252 ms) : 1240, 1264
.   : milestone, 1252,
iast (3.38 ms) : 3341, 3419
.   : milestone, 3380,
iast_FULL (5.969 ms) : 5909, 6029
.   : milestone, 5969,
iast_GLOBAL (3.734 ms) : 3672, 3796
.   : milestone, 3734,
profiling (2.392 ms) : 2366, 2417
.   : milestone, 2392,
tracing (1.9 ms) : 1884, 1916
.   : milestone, 1900,
section candidate
no_agent (1.269 ms) : 1256, 1282
.   : milestone, 1269,
iast (3.242 ms) : 3193, 3290
.   : milestone, 3242,
iast_FULL (6.03 ms) : 5968, 6091
.   : milestone, 6030,
iast_GLOBAL (3.557 ms) : 3501, 3614
.   : milestone, 3557,
profiling (2.267 ms) : 2246, 2287
.   : milestone, 2267,
tracing (1.884 ms) : 1868, 1900
.   : milestone, 1884,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.252 ms [1.24 ms, 1.264 ms]	-
iast	3.38 ms [3.341 ms, 3.419 ms]	2.128 ms (169.9%)
iast_FULL	5.969 ms [5.909 ms, 6.029 ms]	4.717 ms (376.7%)
iast_GLOBAL	3.734 ms [3.672 ms, 3.796 ms]	2.482 ms (198.2%)
profiling	2.392 ms [2.366 ms, 2.417 ms]	1.139 ms (91.0%)
tracing	1.9 ms [1.884 ms, 1.916 ms]	648.078 µs (51.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.269 ms [1.256 ms, 1.282 ms]	-
iast	3.242 ms [3.193 ms, 3.29 ms]	1.973 ms (155.5%)
iast_FULL	6.03 ms [5.968 ms, 6.091 ms]	4.761 ms (375.2%)
iast_GLOBAL	3.557 ms [3.501 ms, 3.614 ms]	2.289 ms (180.4%)
profiling	2.267 ms [2.246 ms, 2.287 ms]	997.945 µs (78.7%)
tracing	1.884 ms [1.868 ms, 1.9 ms]	614.898 µs (48.5%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~cc76233e82, baseline=1.61.0-SNAPSHOT~e307e2c163
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.122 ms) : 17941, 18303
.   : milestone, 18122,
appsec (18.689 ms) : 18501, 18878
.   : milestone, 18689,
code_origins (17.889 ms) : 17711, 18066
.   : milestone, 17889,
iast (18.14 ms) : 17959, 18321
.   : milestone, 18140,
profiling (18.858 ms) : 18670, 19046
.   : milestone, 18858,
tracing (18.039 ms) : 17863, 18215
.   : milestone, 18039,
section candidate
no_agent (19.409 ms) : 19214, 19604
.   : milestone, 19409,
appsec (19.627 ms) : 19429, 19826
.   : milestone, 19627,
code_origins (18.257 ms) : 18076, 18439
.   : milestone, 18257,
iast (17.974 ms) : 17793, 18156
.   : milestone, 17974,
profiling (18.875 ms) : 18690, 19060
.   : milestone, 18875,
tracing (17.56 ms) : 17388, 17732
.   : milestone, 17560,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.122 ms [17.941 ms, 18.303 ms]	-
appsec	18.689 ms [18.501 ms, 18.878 ms]	567.232 µs (3.1%)
code_origins	17.889 ms [17.711 ms, 18.066 ms]	-233.439 µs (-1.3%)
iast	18.14 ms [17.959 ms, 18.321 ms]	18.021 µs (0.1%)
profiling	18.858 ms [18.67 ms, 19.046 ms]	735.742 µs (4.1%)
tracing	18.039 ms [17.863 ms, 18.215 ms]	-83.055 µs (-0.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	19.409 ms [19.214 ms, 19.604 ms]	-
appsec	19.627 ms [19.429 ms, 19.826 ms]	218.608 µs (1.1%)
code_origins	18.257 ms [18.076 ms, 18.439 ms]	-1.152 ms (-5.9%)
iast	17.974 ms [17.793 ms, 18.156 ms]	-1.435 ms (-7.4%)
profiling	18.875 ms [18.69 ms, 19.06 ms]	-533.368 µs (-2.7%)
tracing	17.56 ms [17.388 ms, 17.732 ms]	-1.849 ms (-9.5%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mhlidd/junit_strict_mode
git_commit_date	1775065969	1775278168
git_commit_sha	`e307e2c`	`cc76233`
release_version	1.61.0-SNAPSHOT~e307e2c163	1.61.0-SNAPSHOT~cc76233e82

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1775280407	1775280407
ci_job_id	1568259461	1568259461
ci_pipeline_id	106065531	106065531
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-08f1fde8 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-08f1fde8 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~cc76233e82, baseline=1.61.0-SNAPSHOT~e307e2c163
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.799 s) : 14799000, 14799000
.   : milestone, 14799000,
appsec (14.95 s) : 14950000, 14950000
.   : milestone, 14950000,
iast (18.359 s) : 18359000, 18359000
.   : milestone, 18359000,
iast_GLOBAL (17.751 s) : 17751000, 17751000
.   : milestone, 17751000,
profiling (14.917 s) : 14917000, 14917000
.   : milestone, 14917000,
tracing (14.838 s) : 14838000, 14838000
.   : milestone, 14838000,
section candidate
no_agent (15.402 s) : 15402000, 15402000
.   : milestone, 15402000,
appsec (14.57 s) : 14570000, 14570000
.   : milestone, 14570000,
iast (18.191 s) : 18191000, 18191000
.   : milestone, 18191000,
iast_GLOBAL (17.895 s) : 17895000, 17895000
.   : milestone, 17895000,
profiling (14.952 s) : 14952000, 14952000
.   : milestone, 14952000,
tracing (15.056 s) : 15056000, 15056000
.   : milestone, 15056000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.799 s [14.799 s, 14.799 s]	-
appsec	14.95 s [14.95 s, 14.95 s]	151.0 ms (1.0%)
iast	18.359 s [18.359 s, 18.359 s]	3.56 s (24.1%)
iast_GLOBAL	17.751 s [17.751 s, 17.751 s]	2.952 s (19.9%)
profiling	14.917 s [14.917 s, 14.917 s]	118.0 ms (0.8%)
tracing	14.838 s [14.838 s, 14.838 s]	39.0 ms (0.3%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.402 s [15.402 s, 15.402 s]	-
appsec	14.57 s [14.57 s, 14.57 s]	-832.0 ms (-5.4%)
iast	18.191 s [18.191 s, 18.191 s]	2.789 s (18.1%)
iast_GLOBAL	17.895 s [17.895 s, 17.895 s]	2.493 s (16.2%)
profiling	14.952 s [14.952 s, 14.952 s]	-450.0 ms (-2.9%)
tracing	15.056 s [15.056 s, 15.056 s]	-346.0 ms (-2.2%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~cc76233e82, baseline=1.61.0-SNAPSHOT~e307e2c163
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.489 ms) : 1478, 1501
.   : milestone, 1489,
appsec (3.745 ms) : 3528, 3962
.   : milestone, 3745,
iast (2.271 ms) : 2202, 2339
.   : milestone, 2271,
iast_GLOBAL (2.313 ms) : 2244, 2382
.   : milestone, 2313,
profiling (2.096 ms) : 2041, 2151
.   : milestone, 2096,
tracing (2.078 ms) : 2025, 2131
.   : milestone, 2078,
section candidate
no_agent (1.493 ms) : 1481, 1505
.   : milestone, 1493,
appsec (3.823 ms) : 3601, 4044
.   : milestone, 3823,
iast (2.271 ms) : 2202, 2340
.   : milestone, 2271,
iast_GLOBAL (2.327 ms) : 2257, 2397
.   : milestone, 2327,
profiling (2.158 ms) : 2099, 2218
.   : milestone, 2158,
tracing (2.075 ms) : 2022, 2129
.   : milestone, 2075,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.489 ms [1.478 ms, 1.501 ms]	-
appsec	3.745 ms [3.528 ms, 3.962 ms]	2.255 ms (151.4%)
iast	2.271 ms [2.202 ms, 2.339 ms]	781.242 µs (52.5%)
iast_GLOBAL	2.313 ms [2.244 ms, 2.382 ms]	823.724 µs (55.3%)
profiling	2.096 ms [2.041 ms, 2.151 ms]	606.644 µs (40.7%)
tracing	2.078 ms [2.025 ms, 2.131 ms]	588.852 µs (39.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.493 ms [1.481 ms, 1.505 ms]	-
appsec	3.823 ms [3.601 ms, 4.044 ms]	2.33 ms (156.1%)
iast	2.271 ms [2.202 ms, 2.34 ms]	778.368 µs (52.1%)
iast_GLOBAL	2.327 ms [2.257 ms, 2.397 ms]	833.839 µs (55.9%)
profiling	2.158 ms [2.099 ms, 2.218 ms]	665.362 µs (44.6%)
tracing	2.075 ms [2.022 ms, 2.129 ms]	582.417 µs (39.0%)

mhlidd added 4 commits April 1, 2026 15:38

updating config helper to have descriptive exception in strict mode

4c8153c

init

2199aae

fixing test_base issues and adding drain to confighelpre

3f3d388

suppressforbidden

0b458a3

adding configurations

cc76233

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update JUnit Tests to run with STRICT_TEST mode by default#11045

Update JUnit Tests to run with STRICT_TEST mode by default#11045
mhlidd wants to merge 5 commits intomasterfrom
mhlidd/junit_strict_mode

mhlidd commented Apr 2, 2026

Uh oh!

pr-commenter bot commented Apr 3, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

mhlidd commented Apr 2, 2026

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

pr-commenter bot commented Apr 3, 2026 •

edited

Loading