Skip to content

Add grpc-timeout cross-repo discovery scenario#3

Open
ben-pepper-sg wants to merge 1 commit into
sourcegraph:mainfrom
ajaynz:feature/grpc-timeout-cross-repo-scenario
Open

Add grpc-timeout cross-repo discovery scenario#3
ben-pepper-sg wants to merge 1 commit into
sourcegraph:mainfrom
ajaynz:feature/grpc-timeout-cross-repo-scenario

Conversation

@ben-pepper-sg
Copy link
Copy Markdown

Summary

  • Adds a new grpc-timeout-cross-repo scenario under Understanding existing code that demonstrates hard cross-repo discovery across two real OSS repos: github.com/grpc/grpc-go and github.com/connectrpc/connect-go
  • Scenario prompt asks the agent to trace how context deadlines are encoded onto the wire in both libraries and identify the key architectural difference
  • All code references verified against live repos via Sourcegraph MCP (exact file paths and line numbers)
  • Registered as the first entry in the understand-existing-code scenario list to lead with the most dramatic cross-repo contrast

Why this scenario

The existing cross-repo-discovery scenario is effectively single-repo (vscode Settings Sync). This one is genuinely multi-repo: without MCP the agent physically cannot access connect-go source — every fallback (GOPATH cache, filesystem scan, GitHub API) fails, resulting in a partially-inferred answer that misses the core architectural insight. With MCP it discovers both repos in seconds and traces the full path.

Metrics (from live runs):

Without MCP With MCP
Time 18 min 5.5 min
Cost $3.80 $1.32
Quality 0.18 0.88
Tool calls 85 31

Key finding the scenario surfaces

grpc-go embeds deadline handling inside the HTTP/2 transport layer. connect-go elevates it to a first-class Protocol interface (SetTimeout(*http.Request)), making timeout encoding swappable per protocol. This difference is only visible when reading both codebases side by side — which is exactly what MCP enables.

Test plan

  • npm run dev — new scenario appears in the picker as "Cross-repo protocol comparison"
  • Both agent columns play back with realistic tool call sequences
  • Quality breakdown table renders with all 5 rows
  • Log download buttons work (stub logs in place; replace with live execution logs before shipping)

🤖 Generated with Claude Code

New scenario tracing how RPC deadlines propagate across grpc-go and
connect-go — grounded in real code verified via Sourcegraph MCP. Demonstrates
the 5x quality gap (0.18 → 0.88) when the agent can search both repos
simultaneously vs. being limited to a single local checkout.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant