Skip to content

Updating ConfigHelper to throw error in STRICT_TEST mode and capture undocumented Configs#11031

Open
mhlidd wants to merge 2 commits intomasterfrom
mhlidd/update_strictness_policy
Open

Updating ConfigHelper to throw error in STRICT_TEST mode and capture undocumented Configs#11031
mhlidd wants to merge 2 commits intomasterfrom
mhlidd/update_strictness_policy

Conversation

@mhlidd
Copy link
Copy Markdown
Contributor

@mhlidd mhlidd commented Apr 1, 2026 •
edited
Loading

What Does This Do

Modify StrictnessPolicy.STRICT to StrictnessPolicy.STRICT_TEST, and capturing all configs handled that are not documented. Note that this will only be used in test cases to ensure that all configs are documented in supported-configurations.json. Any tests that use "fake" environment variables in tests can update the test file spec to use StrictnessPolicy.WARNING instead.

This PR also updates StrictnessPolicy.STRICT_TEST to throw an exception including the missing config name instead of returning null. This ensures that we catch all undocumented environment variables that are used.

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

@mhlidd mhlidd marked this pull request as ready for review April 1, 2026 20:01
@mhlidd mhlidd requested a review from a team as a code owner April 1, 2026 20:01
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 1, 2026 •
edited
Loading

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@mhlidd mhlidd changed the title Updating ConfigHelper to throw descriptive error message in Strict mode Updating ConfigHelper to throw descriptive error message in STRICT_TEST mode Apr 1, 2026
@mhlidd mhlidd added type: enhancement Enhancements and improvements comp: config Configuration labels Apr 1, 2026
@pr-commenter
Copy link
Copy Markdown

pr-commenter bot commented Apr 5, 2026

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mhlidd/update_strictness_policy
git_commit_date 1775065969 1775395002
git_commit_sha e307e2c 6ddd1eb
release_version 1.61.0-SNAPSHOT~e307e2c163 1.61.0-SNAPSHOT~6ddd1eb298
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1775396873 1775396873
ci_job_id 1568784449 1568784449
ci_pipeline_id 106115611 106115611
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-mof8ij9k 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-mof8ij9k 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 12 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~6ddd1eb298, baseline=1.61.0-SNAPSHOT~e307e2c163

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.06 s) : 0, 1060016
Total [baseline] (11.061 s) : 0, 11061247
Agent [candidate] (1.055 s) : 0, 1055333
Total [candidate] (11.019 s) : 0, 11018638
section appsec
Agent [baseline] (1.244 s) : 0, 1244236
Total [baseline] (11.14 s) : 0, 11140417
Agent [candidate] (1.242 s) : 0, 1242459
Total [candidate] (11.15 s) : 0, 11150229
section iast
Agent [baseline] (1.223 s) : 0, 1223425
Total [baseline] (11.245 s) : 0, 11245006
Agent [candidate] (1.218 s) : 0, 1218367
Total [candidate] (11.338 s) : 0, 11338394
section profiling
Agent [baseline] (1.181 s) : 0, 1181150
Total [baseline] (11.153 s) : 0, 11152543
Agent [candidate] (1.183 s) : 0, 1182622
Total [candidate] (10.967 s) : 0, 10967381
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.06 s -
Agent appsec 1.244 s 184.219 ms (17.4%)
Agent iast 1.223 s 163.409 ms (15.4%)
Agent profiling 1.181 s 121.134 ms (11.4%)
Total tracing 11.061 s -
Total appsec 11.14 s 79.171 ms (0.7%)
Total iast 11.245 s 183.759 ms (1.7%)
Total profiling 11.153 s 91.297 ms (0.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.055 s -
Agent appsec 1.242 s 187.126 ms (17.7%)
Agent iast 1.218 s 163.035 ms (15.4%)
Agent profiling 1.183 s 127.289 ms (12.1%)
Total tracing 11.019 s -
Total appsec 11.15 s 131.591 ms (1.2%)
Total iast 11.338 s 319.756 ms (2.9%)
Total profiling 10.967 s -51.257 ms (-0.5%) 
gantt
    title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~6ddd1eb298, baseline=1.61.0-SNAPSHOT~e307e2c163

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.202 ms) : 0, 1202
crashtracking [candidate] (1.224 ms) : 0, 1224
BytebuddyAgent [baseline] (633.883 ms) : 0, 633883
BytebuddyAgent [candidate] (630.85 ms) : 0, 630850
AgentMeter [baseline] (29.475 ms) : 0, 29475
AgentMeter [candidate] (29.492 ms) : 0, 29492
GlobalTracer [baseline] (249.38 ms) : 0, 249380
GlobalTracer [candidate] (248.936 ms) : 0, 248936
AppSec [baseline] (31.982 ms) : 0, 31982
AppSec [candidate] (32.011 ms) : 0, 32011
Debugger [baseline] (60.178 ms) : 0, 60178
Debugger [candidate] (59.998 ms) : 0, 59998
Remote Config [baseline] (600.921 µs) : 0, 601
Remote Config [candidate] (591.242 µs) : 0, 591
Telemetry [baseline] (8.075 ms) : 0, 8075
Telemetry [candidate] (8.041 ms) : 0, 8041
Flare Poller [baseline] (9.117 ms) : 0, 9117
Flare Poller [candidate] (8.179 ms) : 0, 8179
section appsec
crashtracking [baseline] (1.197 ms) : 0, 1197
crashtracking [candidate] (1.192 ms) : 0, 1192
BytebuddyAgent [baseline] (659.857 ms) : 0, 659857
BytebuddyAgent [candidate] (659.462 ms) : 0, 659462
AgentMeter [baseline] (12.095 ms) : 0, 12095
AgentMeter [candidate] (12.015 ms) : 0, 12015
GlobalTracer [baseline] (248.022 ms) : 0, 248022
GlobalTracer [candidate] (247.682 ms) : 0, 247682
AppSec [baseline] (183.476 ms) : 0, 183476
AppSec [candidate] (182.919 ms) : 0, 182919
Debugger [baseline] (66.2 ms) : 0, 66200
Debugger [candidate] (65.939 ms) : 0, 65939
Remote Config [baseline] (599.92 µs) : 0, 600
Remote Config [candidate] (603.437 µs) : 0, 603
Telemetry [baseline] (8.582 ms) : 0, 8582
Telemetry [candidate] (8.604 ms) : 0, 8604
Flare Poller [baseline] (3.549 ms) : 0, 3549
Flare Poller [candidate] (3.476 ms) : 0, 3476
IAST [baseline] (24.396 ms) : 0, 24396
IAST [candidate] (24.419 ms) : 0, 24419
section iast
crashtracking [baseline] (1.221 ms) : 0, 1221
crashtracking [candidate] (1.204 ms) : 0, 1204
BytebuddyAgent [baseline] (800.033 ms) : 0, 800033
BytebuddyAgent [candidate] (797.367 ms) : 0, 797367
AgentMeter [baseline] (11.369 ms) : 0, 11369
AgentMeter [candidate] (11.335 ms) : 0, 11335
GlobalTracer [baseline] (239.286 ms) : 0, 239286
GlobalTracer [candidate] (237.652 ms) : 0, 237652
AppSec [baseline] (28.473 ms) : 0, 28473
AppSec [candidate] (32.361 ms) : 0, 32361
Debugger [baseline] (60.273 ms) : 0, 60273
Debugger [candidate] (57.311 ms) : 0, 57311
Remote Config [baseline] (523.5 µs) : 0, 524
Remote Config [candidate] (523.501 µs) : 0, 524
Telemetry [baseline] (14.313 ms) : 0, 14313
Telemetry [candidate] (14.233 ms) : 0, 14233
Flare Poller [baseline] (4.237 ms) : 0, 4237
Flare Poller [candidate] (4.075 ms) : 0, 4075
IAST [baseline] (27.361 ms) : 0, 27361
IAST [candidate] (25.765 ms) : 0, 25765
section profiling
ProfilingAgent [baseline] (93.894 ms) : 0, 93894
ProfilingAgent [candidate] (93.566 ms) : 0, 93566
crashtracking [baseline] (1.169 ms) : 0, 1169
crashtracking [candidate] (1.176 ms) : 0, 1176
BytebuddyAgent [baseline] (689.482 ms) : 0, 689482
BytebuddyAgent [candidate] (690.534 ms) : 0, 690534
AgentMeter [baseline] (9.092 ms) : 0, 9092
AgentMeter [candidate] (9.118 ms) : 0, 9118
GlobalTracer [baseline] (206.474 ms) : 0, 206474
GlobalTracer [candidate] (206.684 ms) : 0, 206684
AppSec [baseline] (32.206 ms) : 0, 32206
AppSec [candidate] (32.581 ms) : 0, 32581
Debugger [baseline] (65.803 ms) : 0, 65803
Debugger [candidate] (65.65 ms) : 0, 65650
Remote Config [baseline] (571.647 µs) : 0, 572
Remote Config [candidate] (565.858 µs) : 0, 566
Telemetry [baseline] (7.829 ms) : 0, 7829
Telemetry [candidate] (7.862 ms) : 0, 7862
Flare Poller [baseline] (3.574 ms) : 0, 3574
Flare Poller [candidate] (3.617 ms) : 0, 3617
Profiling [baseline] (94.472 ms) : 0, 94472
Profiling [candidate] (94.128 ms) : 0, 94128
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~6ddd1eb298, baseline=1.61.0-SNAPSHOT~e307e2c163

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.056 s) : 0, 1055704
Total [baseline] (8.819 s) : 0, 8819204
Agent [candidate] (1.062 s) : 0, 1061746
Total [candidate] (8.822 s) : 0, 8821796
section iast
Agent [baseline] (1.229 s) : 0, 1228519
Total [baseline] (9.581 s) : 0, 9580626
Agent [candidate] (1.224 s) : 0, 1224389
Total [candidate] (9.526 s) : 0, 9526236
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.056 s -
Agent iast 1.229 s 172.815 ms (16.4%)
Total tracing 8.819 s -
Total iast 9.581 s 761.422 ms (8.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.062 s -
Agent iast 1.224 s 162.643 ms (15.3%)
Total tracing 8.822 s -
Total iast 9.526 s 704.44 ms (8.0%) 
gantt
    title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~6ddd1eb298, baseline=1.61.0-SNAPSHOT~e307e2c163

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.193 ms) : 0, 1193
crashtracking [candidate] (1.212 ms) : 0, 1212
BytebuddyAgent [baseline] (632.085 ms) : 0, 632085
BytebuddyAgent [candidate] (635.03 ms) : 0, 635030
AgentMeter [baseline] (29.391 ms) : 0, 29391
AgentMeter [candidate] (29.622 ms) : 0, 29622
GlobalTracer [baseline] (248.178 ms) : 0, 248178
GlobalTracer [candidate] (249.704 ms) : 0, 249704
AppSec [baseline] (31.99 ms) : 0, 31990
AppSec [candidate] (32.168 ms) : 0, 32168
Debugger [baseline] (59.207 ms) : 0, 59207
Debugger [candidate] (59.464 ms) : 0, 59464
Remote Config [baseline] (597.332 µs) : 0, 597
Remote Config [candidate] (598.783 µs) : 0, 599
Telemetry [baseline] (8.066 ms) : 0, 8066
Telemetry [candidate] (8.026 ms) : 0, 8026
Flare Poller [baseline] (8.924 ms) : 0, 8924
Flare Poller [candidate] (9.672 ms) : 0, 9672
section iast
crashtracking [baseline] (1.218 ms) : 0, 1218
crashtracking [candidate] (1.217 ms) : 0, 1217
BytebuddyAgent [baseline] (804.858 ms) : 0, 804858
BytebuddyAgent [candidate] (802.053 ms) : 0, 802053
AgentMeter [baseline] (11.578 ms) : 0, 11578
AgentMeter [candidate] (11.543 ms) : 0, 11543
GlobalTracer [baseline] (239.708 ms) : 0, 239708
GlobalTracer [candidate] (238.989 ms) : 0, 238989
IAST [baseline] (26.779 ms) : 0, 26779
IAST [candidate] (25.996 ms) : 0, 25996
AppSec [baseline] (28.627 ms) : 0, 28627
AppSec [candidate] (31.18 ms) : 0, 31180
Debugger [baseline] (60.309 ms) : 0, 60309
Debugger [candidate] (58.197 ms) : 0, 58197
Remote Config [baseline] (513.817 µs) : 0, 514
Remote Config [candidate] (515.705 µs) : 0, 516
Telemetry [baseline] (14.366 ms) : 0, 14366
Telemetry [candidate] (14.129 ms) : 0, 14129
Flare Poller [baseline] (4.059 ms) : 0, 4059
Flare Poller [candidate] (4.066 ms) : 0, 4066
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mhlidd/update_strictness_policy
git_commit_date 1775065969 1775395002
git_commit_sha e307e2c 6ddd1eb
release_version 1.61.0-SNAPSHOT~e307e2c163 1.61.0-SNAPSHOT~6ddd1eb298
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1775397498 1775397498
ci_job_id 1568784450 1568784450
ci_pipeline_id 106115611 106115611
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-0norbsnv 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-0norbsnv 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 3 performance improvements and 1 performance regressions! Performance is the same for 16 metrics, 16 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:iast_GLOBAL:high_load better
[-264.302µs; -110.361µs] or [-8.900%; -3.716%]		 better
[-664.657µs; -271.434µs] or [-7.989%; -3.263%]		 unstable
[-64.373op/s; +207.811op/s] or [-5.256%; +16.968%]		 2.783ms 7.851ms 1296.469op/s 2.970ms 8.319ms 1224.750op/s
scenario:load:insecure-bank:iast:high_load better
[-305.642µs; -165.464µs] or [-11.292%; -6.113%]		 unsure
[-471.778µs; -82.750µs] or [-6.127%; -1.075%]		 unstable
[-62.464op/s; +230.276op/s] or [-4.677%; +17.242%]		 2.471ms 7.422ms 1419.469op/s 2.707ms 7.700ms 1335.562op/s
scenario:load:petclinic:no_agent:high_load worse
[+0.899ms; +2.418ms] or [+5.142%; +13.836%]		 unstable
[-0.253ms; +3.343ms] or [-0.846%; +11.157%]		 unstable
[-49.520op/s; +6.895op/s] or [-19.099%; +2.659%]		 19.135ms 31.509ms 237.969op/s 17.476ms 29.964ms 259.281op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~6ddd1eb298, baseline=1.61.0-SNAPSHOT~e307e2c163
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.272 ms) : 1259, 1284
.   : milestone, 1272,
iast (3.429 ms) : 3381, 3477
.   : milestone, 3429,
iast_FULL (5.949 ms) : 5889, 6009
.   : milestone, 5949,
iast_GLOBAL (3.747 ms) : 3685, 3808
.   : milestone, 3747,
profiling (2.307 ms) : 2282, 2333
.   : milestone, 2307,
tracing (1.855 ms) : 1839, 1871
.   : milestone, 1855,
section candidate
no_agent (1.247 ms) : 1235, 1260
.   : milestone, 1247,
iast (3.222 ms) : 3178, 3266
.   : milestone, 3222,
iast_FULL (5.843 ms) : 5785, 5901
.   : milestone, 5843,
iast_GLOBAL (3.536 ms) : 3481, 3590
.   : milestone, 3536,
profiling (2.126 ms) : 2107, 2144
.   : milestone, 2126,
tracing (1.913 ms) : 1895, 1931
.   : milestone, 1913,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.272 ms [1.259 ms, 1.284 ms] -
iast 3.429 ms [3.381 ms, 3.477 ms] 2.157 ms (169.6%)
iast_FULL 5.949 ms [5.889 ms, 6.009 ms] 4.677 ms (367.8%)
iast_GLOBAL 3.747 ms [3.685 ms, 3.808 ms] 2.475 ms (194.6%)
profiling 2.307 ms [2.282 ms, 2.333 ms] 1.035 ms (81.4%)
tracing 1.855 ms [1.839 ms, 1.871 ms] 583.332 µs (45.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.247 ms [1.235 ms, 1.26 ms] -
iast 3.222 ms [3.178 ms, 3.266 ms] 1.975 ms (158.3%)
iast_FULL 5.843 ms [5.785 ms, 5.901 ms] 4.596 ms (368.5%)
iast_GLOBAL 3.536 ms [3.481 ms, 3.59 ms] 2.288 ms (183.5%)
profiling 2.126 ms [2.107 ms, 2.144 ms] 878.462 µs (70.4%)
tracing 1.913 ms [1.895 ms, 1.931 ms] 665.423 µs (53.3%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~6ddd1eb298, baseline=1.61.0-SNAPSHOT~e307e2c163
    dateFormat X
    axisFormat %s
section baseline
no_agent (17.996 ms) : 17810, 18183
.   : milestone, 17996,
appsec (19.184 ms) : 18985, 19383
.   : milestone, 19184,
code_origins (17.71 ms) : 17536, 17884
.   : milestone, 17710,
iast (17.742 ms) : 17567, 17917
.   : milestone, 17742,
profiling (18.441 ms) : 18257, 18625
.   : milestone, 18441,
tracing (18.067 ms) : 17885, 18248
.   : milestone, 18067,
section candidate
no_agent (19.615 ms) : 19415, 19816
.   : milestone, 19615,
appsec (18.636 ms) : 18452, 18820
.   : milestone, 18636,
code_origins (17.454 ms) : 17284, 17625
.   : milestone, 17454,
iast (18.184 ms) : 18005, 18364
.   : milestone, 18184,
profiling (18.703 ms) : 18518, 18888
.   : milestone, 18703,
tracing (17.788 ms) : 17615, 17960
.   : milestone, 17788,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 17.996 ms [17.81 ms, 18.183 ms] -
appsec 19.184 ms [18.985 ms, 19.383 ms] 1.188 ms (6.6%)
code_origins 17.71 ms [17.536 ms, 17.884 ms] -285.832 µs (-1.6%)
iast 17.742 ms [17.567 ms, 17.917 ms] -253.928 µs (-1.4%)
profiling 18.441 ms [18.257 ms, 18.625 ms] 444.871 µs (2.5%)
tracing 18.067 ms [17.885 ms, 18.248 ms] 70.607 µs (0.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 19.615 ms [19.415 ms, 19.816 ms] -
appsec 18.636 ms [18.452 ms, 18.82 ms] -979.493 µs (-5.0%)
code_origins 17.454 ms [17.284 ms, 17.625 ms] -2.161 ms (-11.0%)
iast 18.184 ms [18.005 ms, 18.364 ms] -1.431 ms (-7.3%)
profiling 18.703 ms [18.518 ms, 18.888 ms] -912.039 µs (-4.6%)
tracing 17.788 ms [17.615 ms, 17.96 ms] -1.828 ms (-9.3%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mhlidd/update_strictness_policy
git_commit_date 1775065969 1775395002
git_commit_sha e307e2c 6ddd1eb
release_version 1.61.0-SNAPSHOT~e307e2c163 1.61.0-SNAPSHOT~6ddd1eb298
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1775397199 1775397199
ci_job_id 1568784451 1568784451
ci_pipeline_id 106115611 106115611
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-dst3bn9j 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-dst3bn9j 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~6ddd1eb298, baseline=1.61.0-SNAPSHOT~e307e2c163
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.483 ms) : 1472, 1495
.   : milestone, 1483,
appsec (2.516 ms) : 2462, 2571
.   : milestone, 2516,
iast (2.268 ms) : 2199, 2338
.   : milestone, 2268,
iast_GLOBAL (2.315 ms) : 2245, 2385
.   : milestone, 2315,
profiling (2.119 ms) : 2063, 2176
.   : milestone, 2119,
tracing (2.07 ms) : 2016, 2124
.   : milestone, 2070,
section candidate
no_agent (1.482 ms) : 1470, 1493
.   : milestone, 1482,
appsec (3.809 ms) : 3586, 4033
.   : milestone, 3809,
iast (2.261 ms) : 2192, 2330
.   : milestone, 2261,
iast_GLOBAL (2.31 ms) : 2240, 2380
.   : milestone, 2310,
profiling (2.096 ms) : 2041, 2151
.   : milestone, 2096,
tracing (2.066 ms) : 2013, 2119
.   : milestone, 2066,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.483 ms [1.472 ms, 1.495 ms] -
appsec 2.516 ms [2.462 ms, 2.571 ms] 1.033 ms (69.6%)
iast 2.268 ms [2.199 ms, 2.338 ms] 785.004 µs (52.9%)
iast_GLOBAL 2.315 ms [2.245 ms, 2.385 ms] 831.359 µs (56.0%)
profiling 2.119 ms [2.063 ms, 2.176 ms] 635.966 µs (42.9%)
tracing 2.07 ms [2.016 ms, 2.124 ms] 586.922 µs (39.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.482 ms [1.47 ms, 1.493 ms] -
appsec 3.809 ms [3.586 ms, 4.033 ms] 2.328 ms (157.1%)
iast 2.261 ms [2.192 ms, 2.33 ms] 779.481 µs (52.6%)
iast_GLOBAL 2.31 ms [2.24 ms, 2.38 ms] 828.326 µs (55.9%)
profiling 2.096 ms [2.041 ms, 2.151 ms] 614.624 µs (41.5%)
tracing 2.066 ms [2.013 ms, 2.119 ms] 584.217 µs (39.4%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~6ddd1eb298, baseline=1.61.0-SNAPSHOT~e307e2c163
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.253 s) : 15253000, 15253000
.   : milestone, 15253000,
appsec (14.811 s) : 14811000, 14811000
.   : milestone, 14811000,
iast (18.571 s) : 18571000, 18571000
.   : milestone, 18571000,
iast_GLOBAL (17.952 s) : 17952000, 17952000
.   : milestone, 17952000,
profiling (14.805 s) : 14805000, 14805000
.   : milestone, 14805000,
tracing (14.712 s) : 14712000, 14712000
.   : milestone, 14712000,
section candidate
no_agent (15.435 s) : 15435000, 15435000
.   : milestone, 15435000,
appsec (14.71 s) : 14710000, 14710000
.   : milestone, 14710000,
iast (18.188 s) : 18188000, 18188000
.   : milestone, 18188000,
iast_GLOBAL (17.84 s) : 17840000, 17840000
.   : milestone, 17840000,
profiling (14.948 s) : 14948000, 14948000
.   : milestone, 14948000,
tracing (15.271 s) : 15271000, 15271000
.   : milestone, 15271000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.253 s [15.253 s, 15.253 s] -
appsec 14.811 s [14.811 s, 14.811 s] -442.0 ms (-2.9%)
iast 18.571 s [18.571 s, 18.571 s] 3.318 s (21.8%)
iast_GLOBAL 17.952 s [17.952 s, 17.952 s] 2.699 s (17.7%)
profiling 14.805 s [14.805 s, 14.805 s] -448.0 ms (-2.9%)
tracing 14.712 s [14.712 s, 14.712 s] -541.0 ms (-3.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.435 s [15.435 s, 15.435 s] -
appsec 14.71 s [14.71 s, 14.71 s] -725.0 ms (-4.7%)
iast 18.188 s [18.188 s, 18.188 s] 2.753 s (17.8%)
iast_GLOBAL 17.84 s [17.84 s, 17.84 s] 2.405 s (15.6%)
profiling 14.948 s [14.948 s, 14.948 s] -487.0 ms (-3.2%)
tracing 15.271 s [15.271 s, 15.271 s] -164.0 ms (-1.1%)

@mhlidd mhlidd changed the title Updating ConfigHelper to throw descriptive error message in STRICT_TEST mode Updating ConfigHelper to throw error in STRICT_TEST mode and capture undocumented Configs Apr 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@manuel-alvarez-alvarez manuel-alvarez-alvarez Awaiting requested review from manuel-alvarez-alvarez manuel-alvarez-alvarez is a code owner automatically assigned from DataDog/apm-java

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

comp: config Configuration type: enhancement Enhancements and improvements

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@mhlidd