Include all candidate paths for duplicate keys in coverage reports#10997
Conversation
daniel-mohedano
added
type: bug
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 62 metrics, 9 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~a819948127, baseline=1.61.0-SNAPSHOT~65b9b8a192
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.055 s) : 0, 1055083
Total [baseline] (11.033 s) : 0, 11033114
Agent [candidate] (1.059 s) : 0, 1059375
Total [candidate] (11.001 s) : 0, 11001201
section appsec
Agent [baseline] (1.26 s) : 0, 1259699
Total [baseline] (11.185 s) : 0, 11184708
Agent [candidate] (1.257 s) : 0, 1257488
Total [candidate] (11.182 s) : 0, 11181946
section iast
Agent [baseline] (1.225 s) : 0, 1224604
Total [baseline] (11.331 s) : 0, 11331121
Agent [candidate] (1.23 s) : 0, 1229537
Total [candidate] (11.268 s) : 0, 11268082
section profiling
Agent [baseline] (1.183 s) : 0, 1183046
Total [baseline] (11.044 s) : 0, 11044316
Agent [candidate] (1.181 s) : 0, 1180619
Total [candidate] (11.07 s) : 0, 11069781
gantt
title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~a819948127, baseline=1.61.0-SNAPSHOT~65b9b8a192
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.202 ms) : 0, 1202
crashtracking [candidate] (1.208 ms) : 0, 1208
BytebuddyAgent [baseline] (633.095 ms) : 0, 633095
BytebuddyAgent [candidate] (634.197 ms) : 0, 634197
AgentMeter [baseline] (29.4 ms) : 0, 29400
AgentMeter [candidate] (29.528 ms) : 0, 29528
GlobalTracer [baseline] (248.672 ms) : 0, 248672
GlobalTracer [candidate] (249.967 ms) : 0, 249967
AppSec [baseline] (32.028 ms) : 0, 32028
AppSec [candidate] (32.079 ms) : 0, 32079
Debugger [baseline] (60.096 ms) : 0, 60096
Debugger [candidate] (60.107 ms) : 0, 60107
Remote Config [baseline] (600.675 µs) : 0, 601
Remote Config [candidate] (598.558 µs) : 0, 599
Telemetry [baseline] (8.04 ms) : 0, 8040
Telemetry [candidate] (8.043 ms) : 0, 8043
Flare Poller [baseline] (5.808 ms) : 0, 5808
Flare Poller [candidate] (7.521 ms) : 0, 7521
section appsec
crashtracking [baseline] (1.203 ms) : 0, 1203
crashtracking [candidate] (1.211 ms) : 0, 1211
BytebuddyAgent [baseline] (667.258 ms) : 0, 667258
BytebuddyAgent [candidate] (667.519 ms) : 0, 667519
AgentMeter [baseline] (12.148 ms) : 0, 12148
AgentMeter [candidate] (12.13 ms) : 0, 12130
GlobalTracer [baseline] (252.226 ms) : 0, 252226
GlobalTracer [candidate] (251.158 ms) : 0, 251158
AppSec [baseline] (185.294 ms) : 0, 185294
AppSec [candidate] (184.942 ms) : 0, 184942
Debugger [baseline] (67.029 ms) : 0, 67029
Debugger [candidate] (66.359 ms) : 0, 66359
Remote Config [baseline] (614.024 µs) : 0, 614
Remote Config [candidate] (609.765 µs) : 0, 610
Telemetry [baseline] (8.799 ms) : 0, 8799
Telemetry [candidate] (8.746 ms) : 0, 8746
Flare Poller [baseline] (3.695 ms) : 0, 3695
Flare Poller [candidate] (3.586 ms) : 0, 3586
IAST [baseline] (24.875 ms) : 0, 24875
IAST [candidate] (24.719 ms) : 0, 24719
section iast
crashtracking [baseline] (1.231 ms) : 0, 1231
crashtracking [candidate] (1.194 ms) : 0, 1194
BytebuddyAgent [baseline] (801.117 ms) : 0, 801117
BytebuddyAgent [candidate] (804.736 ms) : 0, 804736
AgentMeter [baseline] (11.383 ms) : 0, 11383
AgentMeter [candidate] (11.357 ms) : 0, 11357
GlobalTracer [baseline] (239.114 ms) : 0, 239114
GlobalTracer [candidate] (240.244 ms) : 0, 240244
AppSec [baseline] (31.089 ms) : 0, 31089
AppSec [candidate] (30.905 ms) : 0, 30905
Debugger [baseline] (59.167 ms) : 0, 59167
Debugger [candidate] (59.557 ms) : 0, 59557
Remote Config [baseline] (527.849 µs) : 0, 528
Remote Config [candidate] (521.978 µs) : 0, 522
Telemetry [baseline] (14.244 ms) : 0, 14244
Telemetry [candidate] (14.148 ms) : 0, 14148
Flare Poller [baseline] (4.145 ms) : 0, 4145
Flare Poller [candidate] (4.572 ms) : 0, 4572
IAST [baseline] (25.884 ms) : 0, 25884
IAST [candidate] (26.139 ms) : 0, 26139
section profiling
crashtracking [baseline] (1.196 ms) : 0, 1196
crashtracking [candidate] (1.178 ms) : 0, 1178
BytebuddyAgent [baseline] (691.08 ms) : 0, 691080
BytebuddyAgent [candidate] (689.569 ms) : 0, 689569
AgentMeter [baseline] (9.1 ms) : 0, 9100
AgentMeter [candidate] (9.079 ms) : 0, 9079
GlobalTracer [baseline] (206.744 ms) : 0, 206744
GlobalTracer [candidate] (206.474 ms) : 0, 206474
AppSec [baseline] (32.565 ms) : 0, 32565
AppSec [candidate] (32.466 ms) : 0, 32466
Debugger [baseline] (65.629 ms) : 0, 65629
Debugger [candidate] (65.452 ms) : 0, 65452
Remote Config [baseline] (575.296 µs) : 0, 575
Remote Config [candidate] (564.535 µs) : 0, 565
Telemetry [baseline] (7.776 ms) : 0, 7776
Telemetry [candidate] (7.807 ms) : 0, 7807
Flare Poller [baseline] (3.553 ms) : 0, 3553
Flare Poller [candidate] (3.557 ms) : 0, 3557
ProfilingAgent [baseline] (93.61 ms) : 0, 93610
ProfilingAgent [candidate] (93.404 ms) : 0, 93404
Profiling [baseline] (94.175 ms) : 0, 94175
Profiling [candidate] (93.982 ms) : 0, 93982
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~a819948127, baseline=1.61.0-SNAPSHOT~65b9b8a192
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.054 s) : 0, 1054126
Total [baseline] (8.788 s) : 0, 8787993
Agent [candidate] (1.057 s) : 0, 1056572
Total [candidate] (8.844 s) : 0, 8843809
section iast
Agent [baseline] (1.222 s) : 0, 1221532
Total [baseline] (9.563 s) : 0, 9562694
Agent [candidate] (1.224 s) : 0, 1223711
Total [candidate] (9.5 s) : 0, 9499776
gantt
title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~a819948127, baseline=1.61.0-SNAPSHOT~65b9b8a192
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.188 ms) : 0, 1188
crashtracking [candidate] (1.194 ms) : 0, 1194
BytebuddyAgent [baseline] (632.089 ms) : 0, 632089
BytebuddyAgent [candidate] (633.227 ms) : 0, 633227
AgentMeter [baseline] (29.359 ms) : 0, 29359
AgentMeter [candidate] (29.286 ms) : 0, 29286
GlobalTracer [baseline] (248.037 ms) : 0, 248037
GlobalTracer [candidate] (249.153 ms) : 0, 249153
AppSec [baseline] (31.943 ms) : 0, 31943
AppSec [candidate] (32.048 ms) : 0, 32048
Debugger [baseline] (59.313 ms) : 0, 59313
Debugger [candidate] (59.366 ms) : 0, 59366
Remote Config [baseline] (595.601 µs) : 0, 596
Remote Config [candidate] (597.806 µs) : 0, 598
Telemetry [baseline] (8.1 ms) : 0, 8100
Telemetry [candidate] (8.048 ms) : 0, 8048
Flare Poller [baseline] (7.423 ms) : 0, 7423
Flare Poller [candidate] (7.419 ms) : 0, 7419
section iast
crashtracking [baseline] (1.193 ms) : 0, 1193
crashtracking [candidate] (1.193 ms) : 0, 1193
BytebuddyAgent [baseline] (799.893 ms) : 0, 799893
BytebuddyAgent [candidate] (800.995 ms) : 0, 800995
AgentMeter [baseline] (11.349 ms) : 0, 11349
AgentMeter [candidate] (11.367 ms) : 0, 11367
GlobalTracer [baseline] (238.323 ms) : 0, 238323
GlobalTracer [candidate] (238.739 ms) : 0, 238739
AppSec [baseline] (32.475 ms) : 0, 32475
AppSec [candidate] (30.331 ms) : 0, 30331
Debugger [baseline] (56.714 ms) : 0, 56714
Debugger [candidate] (59.491 ms) : 0, 59491
Remote Config [baseline] (529.546 µs) : 0, 530
Remote Config [candidate] (520.312 µs) : 0, 520
Telemetry [baseline] (14.164 ms) : 0, 14164
Telemetry [candidate] (14.204 ms) : 0, 14204
Flare Poller [baseline] (4.617 ms) : 0, 4617
Flare Poller [candidate] (4.548 ms) : 0, 4548
IAST [baseline] (25.917 ms) : 0, 25917
IAST [candidate] (25.977 ms) : 0, 25977
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 2 performance regressions! Performance is the same for 19 metrics, 15 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~a819948127, baseline=1.61.0-SNAPSHOT~65b9b8a192
dateFormat X
axisFormat %s
section baseline
no_agent (1.255 ms) : 1242, 1267
. : milestone, 1255,
iast (3.318 ms) : 3273, 3363
. : milestone, 3318,
iast_FULL (6.065 ms) : 6003, 6128
. : milestone, 6065,
iast_GLOBAL (3.615 ms) : 3556, 3675
. : milestone, 3615,
profiling (2.362 ms) : 2337, 2388
. : milestone, 2362,
tracing (1.923 ms) : 1906, 1939
. : milestone, 1923,
section candidate
no_agent (1.256 ms) : 1243, 1268
. : milestone, 1256,
iast (3.311 ms) : 3264, 3359
. : milestone, 3311,
iast_FULL (6.031 ms) : 5969, 6092
. : milestone, 6031,
iast_GLOBAL (3.654 ms) : 3594, 3714
. : milestone, 3654,
profiling (2.332 ms) : 2308, 2355
. : milestone, 2332,
tracing (1.917 ms) : 1900, 1934
. : milestone, 1917,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~a819948127, baseline=1.61.0-SNAPSHOT~65b9b8a192
dateFormat X
axisFormat %s
section baseline
no_agent (17.547 ms) : 17373, 17721
. : milestone, 17547,
appsec (18.938 ms) : 18747, 19130
. : milestone, 18938,
code_origins (17.909 ms) : 17729, 18089
. : milestone, 17909,
iast (18.139 ms) : 17961, 18318
. : milestone, 18139,
profiling (19.25 ms) : 19054, 19445
. : milestone, 19250,
tracing (17.936 ms) : 17754, 18117
. : milestone, 17936,
section candidate
no_agent (19.386 ms) : 19194, 19578
. : milestone, 19386,
appsec (18.846 ms) : 18657, 19035
. : milestone, 18846,
code_origins (17.772 ms) : 17597, 17946
. : milestone, 17772,
iast (18.062 ms) : 17883, 18241
. : milestone, 18062,
profiling (19.203 ms) : 19014, 19392
. : milestone, 19203,
tracing (17.868 ms) : 17694, 18042
. : milestone, 17868,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~a819948127, baseline=1.61.0-SNAPSHOT~65b9b8a192
dateFormat X
axisFormat %s
section baseline
no_agent (15.544 s) : 15544000, 15544000
. : milestone, 15544000,
appsec (14.811 s) : 14811000, 14811000
. : milestone, 14811000,
iast (17.833 s) : 17833000, 17833000
. : milestone, 17833000,
iast_GLOBAL (17.813 s) : 17813000, 17813000
. : milestone, 17813000,
profiling (15.345 s) : 15345000, 15345000
. : milestone, 15345000,
tracing (14.698 s) : 14698000, 14698000
. : milestone, 14698000,
section candidate
no_agent (14.942 s) : 14942000, 14942000
. : milestone, 14942000,
appsec (14.625 s) : 14625000, 14625000
. : milestone, 14625000,
iast (18.443 s) : 18443000, 18443000
. : milestone, 18443000,
iast_GLOBAL (18.013 s) : 18013000, 18013000
. : milestone, 18013000,
profiling (14.884 s) : 14884000, 14884000
. : milestone, 14884000,
tracing (14.705 s) : 14705000, 14705000
. : milestone, 14705000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~a819948127, baseline=1.61.0-SNAPSHOT~65b9b8a192
dateFormat X
axisFormat %s
section baseline
no_agent (1.491 ms) : 1479, 1503
. : milestone, 1491,
appsec (3.841 ms) : 3619, 4063
. : milestone, 3841,
iast (2.277 ms) : 2208, 2346
. : milestone, 2277,
iast_GLOBAL (2.326 ms) : 2257, 2396
. : milestone, 2326,
profiling (2.126 ms) : 2070, 2182
. : milestone, 2126,
tracing (2.082 ms) : 2028, 2135
. : milestone, 2082,
section candidate
no_agent (1.491 ms) : 1479, 1502
. : milestone, 1491,
appsec (3.829 ms) : 3604, 4053
. : milestone, 3829,
iast (2.279 ms) : 2210, 2348
. : milestone, 2279,
iast_GLOBAL (2.318 ms) : 2249, 2387
. : milestone, 2318,
profiling (2.542 ms) : 2377, 2707
. : milestone, 2542,
tracing (2.082 ms) : 2029, 2135
. : milestone, 2082,
|
| report = report(testSessionId, testSuiteId, testSpanId, probes.values()); | ||
| return report != null && report.isNotEmpty(); | ||
| } catch (SourceResolutionException e) { | ||
| log.warn( |
There was a problem hiding this comment.
As far as I remember, we automatically send any log that contains an exception to telemetry. Would it be more convenient, perhaps, to include the full stack here (i.e. the exception object) rather than just the message, and lower the log level to debug?
There was a problem hiding this comment.
After refactoring our source paths resolvers to always return a collection of paths to simplify the logic, the exception is not used anymore, so removed the logging in 74ea9f5, but will keep it in mind for the future 👍
| Map<String, List<String>> duplicateTrieKeyPaths = new HashMap<>(); | ||
| for (Map.Entry<String, List<Integer>> entry : duplicateSourceRootIndices.entrySet()) { | ||
| String key = entry.getKey(); | ||
| List<String> paths = new ArrayList<>(); |
There was a problem hiding this comment.
We can pre-size the list with new ArrayList<>(entry.getValue().size()) to avoid reallocations
| roots[e.getValue()] = e.getKey(); | ||
| } | ||
|
|
||
| Map<String, List<String>> duplicateTrieKeyPaths = new HashMap<>(); |
There was a problem hiding this comment.
We can pre-size the map with new HashMap<>(duplicateSourceRootIndices.size() * 4 / 3) to avoid reallocations
| @@ -178,9 +180,18 @@ public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) { | |||
| trieBuilder.put(key, sourceRootIdx); | |||
|
|
|||
| String existingPath = trieKeyToPath.put(key, file.toString()); | |||
There was a problem hiding this comment.
It doesn't seem like we need trieKeyToPath and existingPath any more, do we? Checking for existingSourceRootIdx != null instead should do the trick
Test Environment - sbt-scalatestJob Status: success
|
Test Environment - nebula-release-pluginJob Status: success
|
Test Environment - netflix-zuulJob Status: success
|
Test Environment - pass4sJob Status: success
|
Test Environment - reactive-streams-jvmJob Status: success
|
Test Environment - sonar-kotlinJob Status: success
|
Test Environment - jolokiaJob Status: failed
|
Test Environment - okhttpJob Status: failed
|
Test Environment - spring_bootJob Status: success
|
Test Environment - sonar-javaJob Status: success
|
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
mergequeue build completed successfully, but the github api returned an error while merging the pr. DetailsError: PUT https://api.github.com/repos/DataDog/dd-trace-java/pulls/10997/merge: 405 Merge already in progress [] (Request ID: BE2C:1E3D21:48D59B:12B2F00:69CBC991) FullStacktrace: |
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
There was an unexpected error while creating the working branch: cannot create branch DataDog/dd-trace-java/merge_request:{repository:{owner:"DataDog" name:"dd-trace-java"} queued_at:{seconds:1774963184 nanos:10775000} status_with_reason:STATUS_WITH_REASON_QUEUED priority:200 created_at:{seconds:1774963184 nanos:11910000} pull_request_number:10997 base_branch:"master" head_sha:"a3d3997078719c3e957a5541c97daac7eeb4bafb" id:"7972881f-099a-4ae7-8f05-694440ae9db2" status:STATUS_QUEUED triggered_by:"daniel.mohedano@datadoghq.com" merge_status_workflow_id:"5171baac-2187-4250-9764-97033a0d342e_47" retry_attempt:1} name:"mq-working-branch-master-155f304" temporal_execution_workflow_id:"019d440c-36b7-70da-ba45-578729a37583_287" temporal_execution_run_id:"019d4443-a58d-7003-b0ca-9ee2bd618435" This could indicate that something doesn't work properly with the build system or that this one has reached its maximum capacity. DetailsError: cannot create branch DataDog/dd-trace-java/merge_request:{repository:{owner:"DataDog" name:"dd-trace-java"} queued_at:{seconds:1774963184 nanos:10775000} status_with_reason:STATUS_WITH_REASON_QUEUED priority:200 created_at:{seconds:1774963184 nanos:11910000} pull_request_number:10997 base_branch:"master" head_sha:"a3d3997078719c3e957a5541c97daac7eeb4bafb" id:"7972881f-099a-4ae7-8f05-694440ae9db2" status:STATUS_QUEUED triggered_by:"daniel.mohedano@datadoghq.com" merge_status_workflow_id:"5171baac-2187-4250-9764-97033a0d342e_47" retry_attempt:1} name:"mq-working-branch-master-155f304" temporal_execution_workflow_id:"019d440c-36b7-70da-ba45-578729a37583_287" temporal_execution_run_id:"019d4443-a58d-7003-b0ca-9ee2bd618435" FullStacktrace: |
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
DDCI pipeline didn't start... Please retry. You can get help from #ci-infra-support Error: DDCI pipeline didn't start... Please retry. You can get help from #ci-infra-support (type: DDCIError, retryable: false) DDCI change request: 7767684005639290432 |
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
gh-worker-dd-mergequeue-cf854d
bot
deleted the
daniel.mohedano/coverage-report-duplicate-keys
branch
2 participants
What Does This Do
Motivation
Kotlin-based projects such as Android projects rely on the RepoIndex source path resolution approach to obtain source path information for classes. For Android projects, it's not uncommon to have different implementations for classes for different build variants. During coverage gathering for TIA, these duplicate fully qualified class names effectively aborted the coverage report, making the tests not skippable. This was done to avoid reporting an incorrect class as part of the coverage, causing tests to be skipped when they shouldn't.
With this new approach, instead of invalidating the whole coverage report, we add all the candidate source paths to the coverage report. This avoids the report being incomplete (the test won't be skipped when it shouldn't) at the cost of causing it to not be considered skippable incorrectly some times.
Additional Notes
Local testing using mock server shows all candidates reported:
Another alternative worth exploring in the future is determining the build variant at runtime and using it to infer the correct source path candidate.
test-environment-trigger: skip
Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]
Note: Once your PR is ready to merge, add it to the merge queue by commenting
/merge./merge -ccancels the queue request./merge -f --reason "reason"skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.