feat!: Add per-execution runId, at-most-once tracking, and cross-process tracker resumption#249
Conversation
fe3cd88 to
4dee48f
Compare
|
Tangentially — not introduced by this PR, but surfaced while reviewing it — Failing test that pins the desired behavior (it goes green once the dictionaries are stored as a defensive copy): // ModelConfiguration.Parameters is typed as IReadOnlyDictionary<>, so the
// public contract is "read-only". Even if a consumer downcasts to a
// mutable view, writes through that view must not be visible on
// config.Model.Parameters.
[Fact]
public void DowncastedMutationOfModelParametersMustNotAffectBuiltConfig()
{
var config = LdAiConfig.New().SetModelParam("temperature", LdValue.Of(0.7)).Build();
if (config.Model.Parameters is System.Collections.Generic.IDictionary<string, LdValue> mutable)
{
mutable["temperature"] = LdValue.Of(2.0);
}
Assert.Equal(0.7, config.Model.Parameters["temperature"].AsDouble);
}Two options for the fix:
Happy to file as a separate issue if it's out of scope for this PR. |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 0f3f7d5. Configure here.
0f3f7d5 to
6fdf30e
Compare
Implements the AI SDK billing-spec changes for the tracker:
- Per-execution runId (UUIDv4) is generated at tracker construction and
included in every emitted event so LaunchDarkly can correlate all
metrics emitted by a single run.
- At-most-once event tracking uses Interlocked.CompareExchange on
StrongBox-wrapped nullable slots. Each metric (duration, time-to-
first-token, tokens, feedback, success/error) is recorded exactly
once per tracker even under concurrent calls. Subsequent calls log
a skip warning that includes the track data and points to
CreateTracker for a new run.
- Cross-process tracker resumption via the new ResumptionToken
property and LdAiConfigTracker.FromResumptionToken static method.
The token is a URL-safe Base64 JSON payload (runId, configKey,
variationKey, version) emitted in canonical key order. The ILdAiClient
interface also gains CreateTracker(resumptionToken, context) as the
user-facing resumption entry point. Reconstructed trackers share the
original runId so deferred-feedback flows correlate with the original.
- MetricSummary record exposes the current per-metric state via the
new Summary property on the tracker.
- Validation hardening:
- TrackFeedback throws ArgumentOutOfRangeException for invalid Feedback
- TrackTokens validates Usage before consuming the at-most-once slot
- FromResumptionToken throws ArgumentException for malformed or
incomplete tokens; catches all non-OutOfMemoryException parse failures
- Wire format: empty variationKey is omitted from event track data and
from the resumption token. Empty means "no variation served"
(fallback / default), distinct from "served a variation with key ''".
- Tracker construction: single internal ctor takes individual fields.
Trackers are produced via LdAiConfigBase.CreateTracker (server-side
evaluation, wired by ConfigFactory) or FromResumptionToken
(cross-process resumption).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
6fdf30e to
4fd0cda
Compare
- Default _ldMeta.version to 1 (was 0) when the field is absent on the wire, matches python and js-core. Three sites updated for consistency: ConfigFactory.ParseMeta, ConfigFactory.BuildCompletionFromDefault, and LdAiConfigTracker.ResumptionPayload (so a resumption token without "version" deserializes to 1, not 0). - Default _variationKey to "" in the tracker constructor so a tracker reconstructed from a resumption token without a variationKey holds the same value internally as one created from LdAiConfigBase (which already defaults variationKey ?? ""). - Drop unused `using System;` from LdAiCompletionConfigDefault.cs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ncast ModelConfig.Parameters and .Custom are typed IReadOnlyDictionary<> but were stored as the literal Dictionary<> the caller handed in. The runtime type was mutable; a consumer downcasting to IDictionary<> could mutate the live config. Materialize into ImmutableDictionary<> in the ctor. The cast still succeeds (preserving the IReadOnlyDictionary<> contract) but write members throw NotSupportedException, matching the typed read-only promise. Pre-existing issue surfaced during PR review. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

BEGIN_COMMIT_OVERRIDE
feat!: Add per-execution runId on all AI track event payloads for billing isolation
feat!: Enforce at-most-once tracking — each metric type (duration, tokens, feedback, success/error, time-to-first-token) records once per tracker; duplicates are dropped with a warning
feat!: Add ResumptionToken on ILdAiConfigTracker and ILdAiClient.CreateTracker(resumptionToken, context) for cross-process tracker reconstruction with the original runId
feat: Add MetricSummary property on ILdAiConfigTracker summarizing recorded metrics
END_COMMIT_OVERRIDE
Summary
runId(UUID) included in all track event payloads, enabling per-execution billing isolationCreateTracker()onLdAiConfig: Each call on a config (e.g.LdAiCompletionConfig) returns a new tracker with a freshrunIdand independent tracking state. The returned tracker is always non-null, even when the config is disabledResumptionTokenproperty onILdAiConfigTracker: URL-safe Base64-encoded JSON containing{ runId, configKey, variationKey, version }for cross-process tracker reconstructionCreateTracker(string token, Context context)onILdAiClient: Reconstructs a tracker from a resumption token with the originalrunId, enabling deferred feedback from a different process. Model and provider names are set to empty strings on reconstructionMetricSummaryproperty onILdAiConfigTracker: Snapshot of metrics recorded on this trackerTest plan
CreateTrackerReturnsTrackerWhenDisabled— factory returns a non-null tracker for disabled configsCreateTrackerReturnsNewTrackerWithFreshRunId— each tracker gets a unique runIdCreateTrackerReturnsTrackerWithIndependentTrackingState— at-most-once state is independent per trackerCreateTrackerCanBeCalledMultipleTimes— factory produces multiple independent trackersResumptionTokenContainsExpectedFields— token contains exactly runId, configKey, variationKey, versionResumptionTokenIsUrlSafeBase64— no+,/, or=charactersResumptionTokenIsConsistentAcrossCalls— same token on repeated accessCreateTrackerFromResumptionTokenRoundTrips— full round-trip preserves runIdCreateTrackerFromResumptionTokenSetsEmptyModelAndProvider— reconstructed tracker has empty model/providerCreateTrackerFromInvalidTokenThrows— malformed input throws ArgumentExceptionCreateTrackerFromNullTokenThrows— null input throws ArgumentNullExceptionCreateTrackerFromTokenMissingRunIdThrows— incomplete payload throws ArgumentException🤖 Generated with Claude Code
Note
Medium Risk
Changes public tracking semantics (runId on events, duplicate metrics ignored) and default version behavior, which can affect downstream analytics and billing correlation; resumption token parsing is a new trust boundary for cross-process flows.
Overview
AI config trackers are now scoped to a single run with a UUID
runIdon every track payload,ResumptionToken/ILdAiClient.CreateTracker(token, context)for cross-process reuse of that run, and aMetricSummarysnapshot of what was recorded.LdAiConfigTrackeris rebuilt around explicit fields (not the whole config): eachCreateTracker()gets a newrunId; duplicateTrack*calls are dropped (with warnings) using atomic slots, including shared success/error; empty token usage does not consume the tokens slot. Track metadata addsrunId, omits emptyvariationKey, and resumption tokens are URL-safe Base64 JSON without model/provider.Smaller contract tweaks: caller defaults use
version: 1; missing_ldMeta.versiondefaults to 1;ModelConfigstores parameters inImmutableDictionaryso downcasts cannot mutate maps.Reviewed by Cursor Bugbot for commit f7488be. Bugbot is set up for automated code reviews on this repo. Configure here.