feat: support "draft" as a first-class spec version target by pcarleton · Pull Request #255 · modelcontextprotocol/conformance

pcarleton · 2026-04-24T11:58:12Z

Closes #253.

Problem

SEP authors testing against the in-progress spec can't run the full suite: the initialize check rejects draft protocolVersion strings, and --spec-version draft only selects the handful of explicitly draft-tagged scenarios (skipping initialize and the rest of the core suite). Tagging core scenarios with 'draft' is blocked by the spec-version isolation test.

Raised by @mikekistler in #conformance-testing-wg while testing SEP-2243.

Design

SpecVersion is now always a wire protocolVersion string — the same value an SDK sends in initialize and the value --spec-version accepts:

export const DRAFT_PROTOCOL_VERSION = 'DRAFT-2026-v1';
export type SpecVersion = DatedSpecVersion | typeof DRAFT_PROTOCOL_VERSION;

DRAFT_PROTOCOL_VERSION mirrors LATEST_PROTOCOL_VERSION in the spec repo's schema/draft/schema.ts (which has cycled DRAFT-2025-v1 → -v2 → -v3 → DRAFT-2026-v1). When the spec repo bumps it, this constant gets a matching bump and a conformance release — consumers updating to that release see the new value in the type, in --spec-version validation, and in scenario tags simultaneously. Stale draft strings are rejected, which is a useful signal that the SDK and conformance package are out of sync.

The CLI also accepts --spec-version draft as a one-line alias that resolves to DRAFT_PROTOCOL_VERSION, purely for ergonomics; the literal 'draft' does not appear in the type system.

A '2026-06-DRAFT' form was considered and rejected: the spec repo doesn't use it anywhere and the release date isn't known in advance.

Changes

types.ts: add DATED_SPEC_VERSIONS, LATEST_SPEC_VERSION, DRAFT_PROTOCOL_VERSION, and NEGOTIABLE_PROTOCOL_VERSIONS. SpecVersion = DatedSpecVersion | typeof DRAFT_PROTOCOL_VERSION. 'extension' moved to a separate ScenarioSpecTag type — it's a scenario category, not a point on the version timeline.
checks/client.ts / scenarios/client/initialize.ts: both use the shared NEGOTIABLE_PROTOCOL_VERSIONS constant; the mock server accepts and echoes DRAFT_PROTOCOL_VERSION.
scenarios/index.ts: new matchesSpecVersion() helper. Selecting the draft version resolves to latest-dated ∪ draft-tagged, so SEP authors can run the full suite without retagging core scenarios. resolveSpecVersion adds the 'draft' CLI alias. --spec-version extension is no longer valid; extension scenarios are reachable via --suite extensions.
runner/client.ts / index.ts: forward the resolved --spec-version value to the client process as MCP_CONFORMANCE_PROTOCOL_VERSION. Example clients can use it directly as their protocolVersion; SDKs that hard-code their version can ignore it.
Draft-tagged scenarios (offline-access, resource-mismatch, sep-2164-resource-not-found): now reference the DRAFT_PROTOCOL_VERSION constant, so a draft revision bump is a one-line change in types.ts.
tier-check: derives tier-scoring versions from DATED_SPEC_VERSIONS; draft and extension remain non-scoring. A tier-check run at the draft version scores against latest ∪ draft (i.e., "would this SDK pass tier N if draft shipped today").
Tests: superset relationship and tag-level isolation asserted separately; positive test for DRAFT_PROTOCOL_VERSION acceptance, negative tests for stale 'DRAFT-2025-v1' and bare 'draft' on the wire; 'draft' CLI alias resolution; extension scenarios excluded from all --spec-version selections.
README documents --spec-version and the new env var.

Testing

$ npx vitest run
Test Files  8 passed (8)
     Tests  116 passed (116)

$ node dist/index.js list --spec-version DRAFT-2025-v1
Unknown spec version: DRAFT-2025-v1
Valid versions: 2025-03-26, 2025-06-18, 2025-11-25, DRAFT-2026-v1 (or 'draft' as an alias for DRAFT-2026-v1)

Typecheck and lint clean.

Follow-ups (not in this PR)

Replace specVersions list with introducedIn/removedIn range tagging #256 — range-based tagging: replace per-scenario specVersions: [...] lists with introducedIn / optional removedIn. The superset model in this PR can't express a SEP that tightens or removes an existing requirement (e.g., SEP-986 narrowed valid tool-name format) — there's no subtraction arm. matchesSpecVersion() is the seam where range logic slots in.
Migrating remaining hard-coded '2025-11-25' literals in server scenarios to LATEST_SPEC_VERSION.

- Add LATEST_SPEC_VERSION and DATED_SPEC_VERSIONS constants so the next spec release is a one-line change in types.ts. - Accept "draft" as a valid protocolVersion in the initialize check and mock-server response. - --spec-version draft now selects latest-dated scenarios plus draft-tagged ones, so SEP authors can run the full suite against an SDK tracking the in-progress spec without retagging core scenarios. - Forward --spec-version to the client process via MCP_CONFORMANCE_SPEC_VERSION so SDK examples can pick the matching protocolVersion. Closes #253

pkg-pr-new · 2026-04-24T11:58:32Z

Open in StackBlitz

npx https://pkg.pr.new/@modelcontextprotocol/conformance@255

commit: 7a87bf7

felixweinberger

LGTM, optional nit on consolidating a single constant instead of having 2 copies

felixweinberger · 2026-04-24T12:33:50Z

+    const VALID_VERSIONS = [
+      '2025-06-18',
+      LATEST_SPEC_VERSION,
+      DRAFT_PROTOCOL_VERSION
+    ];


could consider consolildating a VALID_VERSIONS constant in types.ts instead of having 2 copies here and in client.ts

felixweinberger · 2026-04-24T12:40:56Z

+  // could reuse the filter plumbing. It has no corresponding wire
+  // protocolVersion. Split it out of this type when moving to
+  // introducedIn/removedIn tagging.
+  if (version === 'extension') return undefined;


feels like something worth cleaning up, spec vs protocolVersion might get more confusing over time the longer we have both of these 🤔

👍 we need a different way to refer to extensions, it doesn't super make sense to run "all extensions" as a selector. ties a bit into my comment on #114 about making it easier to selectively run scenarios via a config file

mikekistler · 2026-04-27T15:28:31Z

+
+// Mirrors LATEST_PROTOCOL_VERSION in the spec repo's schema/draft/schema.ts.
+// Bump when that constant changes.
+export const DRAFT_PROTOCOL_VERSION = 'DRAFT-2026-v1';


I'm confused. What protocol version should we be using for conformance tests of draft features? This one or "draft"? If it is this one, then why the special casing of "draft" for the --spec-version parameter?

fair question. I was thinking that draft would be a convenient alias, but looking at it again using the actual draft spec version will mean it's an easier string replace when we finalize the actual spec, and the alias just adds another indirection.

I've reworked it to be this one everywhere. so you can pass --spec-version DRAFT-2026-v1 and it will work. It will also work to pass --spec-version draft as a convenience, but tagging scenarios you need to use the explicit one now (not draft).

@felixweinberger

…OL_VERSIONS; split out 'extension' Addresses review feedback from @felixweinberger and @mikekistler: - SpecVersion is now always a wire protocolVersion string (DatedSpecVersion | typeof DRAFT_PROTOCOL_VERSION). The separate 'draft' tag literal is gone from the type system; 'draft' survives only as a CLI input alias in resolveSpecVersion. Removes the tag-vs-wire confusion and the specVersionToProtocolVersion mapping. - NEGOTIABLE_PROTOCOL_VERSIONS in types.ts is the single source for what the mock server accepts on initialize (was duplicated in checks/client.ts and scenarios/client/initialize.ts). - 'extension' moved out of SpecVersion into a separate ScenarioSpecTag type. --spec-version extension is no longer valid; extension scenarios remain reachable via --suite extensions. - Draft-tagged scenarios now use the DRAFT_PROTOCOL_VERSION constant so a draft revision bump is a one-line change in types.ts.

felixweinberger

Thanks for the changes!

mikekistler

Looks good! 👍

Just confirmed that I can layer my tests for SEP-2243 on top of this and successfully test the MCP C# reference implementation.

pcarleton mentioned this pull request Apr 24, 2026

Replace specVersions list with introducedIn/removedIn range tagging #256

Open

pcarleton requested a review from mikekistler April 24, 2026 12:02

felixweinberger previously approved these changes Apr 24, 2026

View reviewed changes

mikekistler reviewed Apr 27, 2026

View reviewed changes

pcarleton dismissed felixweinberger’s stale review via 7a87bf7 April 28, 2026 12:37

pcarleton requested a review from felixweinberger April 28, 2026 13:04

felixweinberger approved these changes Apr 28, 2026

View reviewed changes

mikekistler approved these changes Apr 28, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: support "draft" as a first-class spec version target#255

feat: support "draft" as a first-class spec version target#255
pcarleton wants to merge 2 commits intomainfrom
paulc/draft-spec-version

pcarleton commented Apr 24, 2026 •

edited

Loading

Uh oh!

pkg-pr-new Bot commented Apr 24, 2026 •

edited

Loading

Uh oh!

felixweinberger left a comment

Uh oh!

felixweinberger Apr 24, 2026

Uh oh!

felixweinberger Apr 24, 2026

Uh oh!

pcarleton Apr 24, 2026

Uh oh!

mikekistler Apr 27, 2026

Uh oh!

pcarleton Apr 28, 2026

Uh oh!

felixweinberger left a comment

Uh oh!

mikekistler left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

pcarleton commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Design

Changes

Testing

Follow-ups (not in this PR)

Uh oh!

pkg-pr-new Bot commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

felixweinberger left a comment

Choose a reason for hiding this comment

Uh oh!

felixweinberger Apr 24, 2026

Choose a reason for hiding this comment

Uh oh!

felixweinberger Apr 24, 2026

Choose a reason for hiding this comment

Uh oh!

pcarleton Apr 24, 2026

Choose a reason for hiding this comment

Uh oh!

mikekistler Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

pcarleton Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

felixweinberger left a comment

Choose a reason for hiding this comment

Uh oh!

mikekistler left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pcarleton commented Apr 24, 2026 •

edited

Loading

pkg-pr-new Bot commented Apr 24, 2026 •

edited

Loading