docs: clarify BYOK + Custom Inference request path and data flow

[hongyi-chen]

Summary

Clarifies the request path and data-flow framing for both BYOK and Custom Inference endpoints in our docs, in response to warpdotdev/warp#11681 and the follow-up triage thread.

The previous wording on both pages combined two claims that aren't equivalent:

1. Storage: API keys are stored locally (true and unchanged).
2. Transit: requests are routed "directly" to the model provider — that part was misleading. The Warp Agent harness is server-hosted, so requests do transit Warp's backend; the key is passed in-flight per request and used to authenticate the call from warp-server, not from the client.

Both the recent r/warpdotdev complaint and issue #11681 (Custom Inference) traced back to this framing.

Per Daniel's clarification in the linked thread, the narrow durable claim Warp wants to stand behind is: API keys are never synced or stored on Warp's servers.

Confirmed against warp-server:

• logic/ai/llm/anthropic/util/util.go:1032-1034 — server overrides the Anthropic SDK API key with the user-provided one per request.
• logic/ai/llm/user_api_keys/util.go:7 — keys arrive in the request payload as Request_Settings_ApiKeys.
• logic/ai/llm/llm_role.go:723 — server-side model routing applies BYOK preferences via WithApiKeyConfigApplied.
• logic/ai/llm/custom_endpoint/client.go:14-21 — the OpenAI-compatible client is constructed server-side with option.WithAPIKey(hostConfig.CustomEndpointAPIKey()) and option.WithBaseURL(...CustomEndpointBaseURL()); both come in on the request.

Changes

src/content/docs/agent-platform/inference/bring-your-own-api-key.mdx

• Intro paragraph: dropped "access models directly" wording.
• How BYOK works section: replaced the single "directly route your agent requests" line with an explicit 3-step data flow (harness assembles request → key authenticates the call in-flight → response streams back), and clarified that keys live in-memory only for the duration of each request.
• Headline storage claim now uses Daniel's wording: "never synced or stored on Warp's servers".
• Added a Why does the request route through Warp's backend? note explaining the server-side harness (same runtime as Agent Mode with Warp-billed models).
• ZDR section: added a sentence noting BYOK request bodies transit Warp's backend but are not retained, used for training, or logged for analytics — same posture as Warp-billed traffic. Scoped the existing "data retention policies depend on..." bullet to be explicit it's about the provider side.
• Tightened the diagram alt text from "directly through your provider API key" → "authenticates BYOK agent requests with your provider API key".
• Fixed a stale anchor: the ZDR section linked to #how-does-byok-work but the heading on main is now How BYOK works (slug #how-byok-works).

src/content/docs/agent-platform/inference/custom-inference-endpoint.mdx

• Key features: rewrote the "Local configuration" bullet — API keys are stored locally on the device and never synced or stored on Warp's servers.
• How it works: replaced the blanket "never synced to Warp's servers" wording with an explicit 3-step request flow mirroring the BYOK rewrite.
• Added a Why does the request route through Warp's backend? callout matching the BYOK page, explicitly cross-linking to BYOK so readers see the consistent posture.
• ZDR section: added the same "request bodies transit Warp's backend but aren't used for training" framing as BYOK, and scoped the existing retention bullets to the provider side.

Additional context

• See the implementation plan for the research summary and the breakdown that informed this update.
• Conversation thread.
• No structural changes (no new top-level sections, no terminology drift), no sidebar config edits, no redirects required.
• Internal link checker: 0 broken links (python3 .agents/skills/check_for_broken_links/check_links.py --internal-only → 2702 internal links checked, 0 broken).

Conversation: https://staging.warp.dev/conversation/c3a085dc-4658-47c2-9908-e7f56672872f
Run: https://oz.staging.warp.dev/runs/019e665f-0f94-780b-9764-04bdbd28a24b
Plans:

• Expand PR #138 to address Custom Inference Endpoint privacy claims

This PR was generated with Oz.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
docs	Ready	Preview, Comment	May 26, 2026 11:08pm

[oz-for-oss]

@hongyi-chen

I'm starting a first review of this pull request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

[oz-for-oss]

Overview

This PR clarifies the BYOK documentation by explaining that BYOK agent requests transit Warp's backend and that the user-provided provider key is used in flight. The added data-flow description is useful, but one new privacy/retention statement is broader than the existing privacy documentation supports.

Concerns

• The new ZDR section says Warp does not retain the BYOK request body or log it for analytics, but the privacy documentation describes account-level telemetry and plan settings that can affect AI interaction collection. This should be scoped before merge.

Verdict

Found: 0 critical, 1 important, 0 suggestions

Request changes

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz