docs: add tracemetrics dataset guidance and validate aggregate format#636
Merged
docs: add tracemetrics dataset guidance and validate aggregate format#636
Conversation
Add comprehensive documentation for the tracemetrics dataset: - Agent guidance with query format, parameter table, and examples - Common dashboard widget mistakes section - AGENTS.md lore entry for tracemetrics gotchas Add validation in validateAggregateNames() that rejects span-style aggregates (e.g., count(), p50(span.duration)) when --dataset tracemetrics is used, with a helpful error message showing the correct comma-separated format: aggregation(value,name,type,unit). Key lessons encoded: - tracemetrics uses aggregation(value,metric_name,metric_type,unit) NOT the MRI format d:custom/name@unit used by the metrics dataset - The unit must match what the SDK emits (none if unspecified) - --group-by always requires --limit - --sort must reference a field present in --query - Span attributes cannot be used as aggregate fields
Contributor
Semver Impact of This PR🟢 Patch (bug fixes) 📋 Changelog PreviewThis is how your changes will appear in the changelog. New Features ✨Telemetry
Other
Bug Fixes 🐛Telemetry
Other
Documentation 📚
Internal Changes 🔧
🤖 This preview updates automatically when you update the PR. |
Contributor
|
Contributor
Codecov Results 📊✅ 129 passed | Total: 129 | Pass Rate: 100% | Execution Time: 0ms 📊 Comparison with Base Branch
✨ No test changes detected All tests are passing successfully. ✅ Patch coverage is 85.71%. Project has 1353 uncovered lines. Files with missing lines (1)
Coverage diff@@ Coverage Diff @@
## main #PR +/-##
==========================================
- Coverage 95.56% 95.54% -0.02%
==========================================
Files 205 205 —
Lines 30270 30352 +82
Branches 0 0 —
==========================================
+ Hits 28928 28999 +71
- Misses 1342 1353 +11
- Partials 0 0 —Generated by Codecov Action |
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
…guidance to widget help Remove the legacy 'metrics' dataset (MRI-based, d:custom/name@unit) from WIDGET_TYPES, DATASET_SUPPORTED_DISPLAY_TYPES, widget help text, agent guidance, and tests. The 'tracemetrics' dataset with its comma-separated format is the correct choice for SDK v10+ custom metrics. Move dataset descriptions and tracemetrics query format docs into the widget command's fullDescription so both humans and agents see it via 'sentry dashboard widget --help'.
…rences --query Add client-side validation that catches two common API errors early: 1. validateGroupByRequiresLimit: Widgets with --group-by must include --limit. The Sentry API rejects grouped widgets without a limit. Only fires for explicit --group-by, not auto-defaulted columns (e.g., issue dataset auto-defaults columns to ['issue']). 2. validateSortReferencesAggregate: The --sort expression must reference an aggregate present in --query. Prevents 400 errors from the API when sort references a field not in the widget's aggregates. Both validations run in the add path (buildWidgetFromFlags) and edit path (validateQueryConstraints), with the edit path only checking when the user actively changes query/group-by/sort flags.
…elds Add warnUnknownAggregateFields() that logs a warning when an aggregate field doesn't match known aggregatable span fields (span.duration, span.self_time, http.*, cache.* measurements). Span attributes like dsn.files_collected or resolve.method are key-value metadata that cannot be aggregated — they should be used in --where or --group-by. Uses a warning (not error) because measurements are project-specific and the known-good list may not be exhaustive.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
- Fix validateQueryConstraints false positive: only check group-by+limit when user explicitly passes --group-by, not when merely changing --query on an existing grouped widget (which may have auto-defaulted columns) - Fix orphaned JSDoc: move buildWidgetFromFlags doc comment to its actual function definition after the validation helpers were inserted
…from existing widget
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
…fields Functions like count_unique(transaction) and any(span.op) accept non-numeric columns and should not trigger the span-attribute warning. Only numeric aggregates (avg, sum, p50-p100, min, max, percentile) require measurement fields.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tracemetricsdataset to prevent future agents from using the wrong query formatvalidateAggregateNames()guard that rejects span-style aggregates when--dataset tracemetricsis usedContext
While building the CLI Performance dashboard, several mistakes were made with custom metrics widgets that could have been prevented with better docs and validation:
--dataset metricswith MRI format (d:custom/name@unit) instead of--dataset tracemetricswith comma-separated format (aggregation(value,name,type,unit)) — SDK v10+ emitstrace_metricitems, not standalone custom metrics@millisecondwhen SDK emits with no unit →@none) — caused "Internal Error" on every metrics widget--limiton grouped widgets — API rejects grouped queries without limit--sortreferencing fields not in--query— caused 400 errors (e.g., sorting bycount()after removing it from aggregates)avg:dsn.files_collected) — span attributes are metadata, not aggregatable measurementsChanges
docs/src/content/docs/agent-guidance.mdsrc/types/dashboard.tsisTracemetricsAggregate()validator + tracemetrics branch invalidateAggregateNames()AGENTS.mdplugins/sentry-cli/skills/sentry-cli/SKILL.md