Dual-path retry: exponential backoff + CDN-driven httpConfig support#143
Open
MichaelGHSeg wants to merge 4 commits into
Open
Dual-path retry: exponential backoff + CDN-driven httpConfig support#143MichaelGHSeg wants to merge 4 commits into
MichaelGHSeg wants to merge 4 commits into
Conversation
- HTTPClient: dual-path retry loop (429+Retry-After vs counted exponential backoff), configurable retry properties (MaxRetries, MaxTotalBackoffDuration, MaxRateLimitDuration, BackoffEnabled, RateLimitEnabled, BaseBackoffMs, MaxBackoffMs, StatusCodeOverrides), X-Retry-Count header on retries - Configuration: add MaxRetries, MaxTotalBackoffDuration, MaxRateLimitDuration constructor params - EventPipeline: wire config retry params into HTTPClient at construction; expose _httpClient as internal - SegmentDestination: parse httpConfig from CDN settings response and apply to HTTPClient at runtime - UnityHTTPClient: update DoPost signature to match new abstract base - Tests.csproj: add net10.0 target for local dev (net6.0 runtime no longer installed)
HTTPClient.cs: - Rewrite Upload() with dual-path retry: 429+Retry-After uses uncounted rate-limit path; other retryable errors use counted exponential backoff - Add configurable properties: MaxRetries, MaxRateLimitRetries, BackoffEnabled, RateLimitEnabled, BaseBackoffMs, MaxBackoffMs, MaxRetryAfterCapSeconds, StatusCodeOverrides - Return true (discard) on all terminal failure paths so EventPipeline removes the batch file instead of re-uploading indefinitely - Accept Retry-After: 0 as valid (sleep 0ms, still counts as rate-limit) SegmentDestination.cs: - Read httpConfig from CDN settings and apply backoffConfig/rateLimitConfig - Use CultureInfo.InvariantCulture for double parsing - Map maxRetryInterval to MaxRetryAfterCapSeconds (per-request cap) e2e-cli: - Add CapturingLogger that only captures final-failure messages - Use enrichment callbacks for userId instead of pre-identify - Add AUTO_SETTINGS 2s wait for settings to arrive before first upload - Enable retry-settings test suite
…ides - Clamp ParseRetryAfter minimum to 1 second to prevent busy-loop when server sends Retry-After: 0 or MaxRetryAfterCapSeconds is misconfigured - Replace hardcoded status code list in ParseStatusCodeOverrides with dynamic key enumeration (matches analytics-kotlin approach), so CDN overrides for any valid HTTP status code (100-599) are respected
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rewrites
HTTPClient.Upload()with a structured dual-path retry system and adds support for reading retry configuration from CDN settings (integrations["Segment.io"].httpConfig).Retry logic (
HTTPClient.cs)MaxRateLimitRetriesandMaxRateLimitDuration(default 12h).Retry-After: 0is treated as a 1-second floor to prevent busy-loops.MaxRetriesandMaxTotalBackoffDuration(default 12h).trueon all terminal failure paths soEventPipelineremoves the batch file instead of re-uploading indefinitely.MaxRetries,MaxRateLimitRetries,BackoffEnabled,RateLimitEnabled,BaseBackoffMs,MaxBackoffMs,MaxRetryAfterCapSeconds,StatusCodeOverrides.X-Retry-Countheader on retry attempts.CDN-driven config (
SegmentDestination.cs)httpConfig.backoffConfigandhttpConfig.rateLimitConfigfrom CDN settings and applies them to the HTTP client after settings arrive.statusCodeOverridesuses dynamic key enumeration (matching the analytics-kotlin approach) instead of a hardcoded status code list — any valid HTTP status code (100–599) in the CDN config is respected.CultureInfo.InvariantCulturefordouble.TryParseto handle non-English locales correctly.maxRetryIntervaltoMaxRetryAfterCapSeconds(per-request sleep cap, not total budget).E2E cli (
e2e-cli/)CapturingLoggerfilters to final-failure messages only ("Retries exhausted", "Max total backoff") to avoid false failures from transient per-attempt errors.userIdinjection instead of pre-callingIdentify.new Analytics()whenAUTO_SETTINGS=trueso CDN settings arrive before the first upload.retryandretry-settingstest suites.Test plan
dotnet testpassesbasic,retry,retry-settingssuites pass (68/68)