fix(models): default fable-5 to adaptive thinking when no effort is set#2618
fix(models): default fable-5 to adaptive thinking when no effort is set#2618posthog[bot] wants to merge 1 commit into
Conversation
claude-fable-5 generations without a configured effort level were
deterministically rejected by Anthropic with
`invalid_request_error: "thinking.type.disabled" is not supported for
this model`. With no effort set the SDK falls back to
`thinking: { type: "disabled" }`, which fable-5 (adaptive-only) rejects.
Gate a new `requiresAdaptiveThinking` capability flag in
session/models.ts and, in the effort→options mapping in
session/options.ts, default such models to a "medium" effort so the SDK
requests `thinking: { type: "adaptive" }` even when the user has not
chosen an effort level. Other models keep their existing behavior.
Generated-By: PostHog Code
Task-Id: 2bd7cca9-5bcf-4889-98cc-6abbd91de2f7
|
React Doctor found no issues in the changed files. 🎉 Reviewed by React Doctor for commit |
Prompt To Fix All With AIFix the following 1 code review issue. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 1
packages/agent/src/adapters/claude/session/models.test.ts:67-73
The team prefers parameterised tests, and this test checks the same `requiresAdaptiveThinking` function against five different model IDs with expected booleans — a perfect fit for `it.each`. Without parameterisation, a future maintainer adding a new model would need to know to add a new `expect` line inside an existing test rather than extend a table, and a single failure obscures which model caused it.
```suggestion
it.each([
["claude-fable-5", true],
["claude-opus-4-8", false],
["claude-opus-4-7", false],
["claude-sonnet-4-6", false],
["claude-haiku-4-5", false],
] as const)(
"requiresAdaptiveThinking(%s) === %s",
(modelId, expected) => {
expect(requiresAdaptiveThinking(modelId)).toBe(expected);
},
);
```
Reviews (1): Last reviewed commit: "fix(models): default fable-5 to adaptive..." | Re-trigger Greptile |
| it("flags only models that reject disabled thinking as requiring adaptive thinking", () => { | ||
| expect(requiresAdaptiveThinking("claude-fable-5")).toBe(true); | ||
| expect(requiresAdaptiveThinking("claude-opus-4-8")).toBe(false); | ||
| expect(requiresAdaptiveThinking("claude-opus-4-7")).toBe(false); | ||
| expect(requiresAdaptiveThinking("claude-sonnet-4-6")).toBe(false); | ||
| expect(requiresAdaptiveThinking("claude-haiku-4-5")).toBe(false); | ||
| }); |
There was a problem hiding this comment.
The team prefers parameterised tests, and this test checks the same
requiresAdaptiveThinking function against five different model IDs with expected booleans — a perfect fit for it.each. Without parameterisation, a future maintainer adding a new model would need to know to add a new expect line inside an existing test rather than extend a table, and a single failure obscures which model caused it.
| it("flags only models that reject disabled thinking as requiring adaptive thinking", () => { | |
| expect(requiresAdaptiveThinking("claude-fable-5")).toBe(true); | |
| expect(requiresAdaptiveThinking("claude-opus-4-8")).toBe(false); | |
| expect(requiresAdaptiveThinking("claude-opus-4-7")).toBe(false); | |
| expect(requiresAdaptiveThinking("claude-sonnet-4-6")).toBe(false); | |
| expect(requiresAdaptiveThinking("claude-haiku-4-5")).toBe(false); | |
| }); | |
| it.each([ | |
| ["claude-fable-5", true], | |
| ["claude-opus-4-8", false], | |
| ["claude-opus-4-7", false], | |
| ["claude-sonnet-4-6", false], | |
| ["claude-haiku-4-5", false], | |
| ] as const)( | |
| "requiresAdaptiveThinking(%s) === %s", | |
| (modelId, expected) => { | |
| expect(requiresAdaptiveThinking(modelId)).toBe(expected); | |
| }, | |
| ); |
Context Used: Do not attempt to comment on incorrect alphabetica... (source)
Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/agent/src/adapters/claude/session/models.test.ts
Line: 67-73
Comment:
The team prefers parameterised tests, and this test checks the same `requiresAdaptiveThinking` function against five different model IDs with expected booleans — a perfect fit for `it.each`. Without parameterisation, a future maintainer adding a new model would need to know to add a new `expect` line inside an existing test rather than extend a table, and a single failure obscures which model caused it.
```suggestion
it.each([
["claude-fable-5", true],
["claude-opus-4-8", false],
["claude-opus-4-7", false],
["claude-sonnet-4-6", false],
["claude-haiku-4-5", false],
] as const)(
"requiresAdaptiveThinking(%s) === %s",
(modelId, expected) => {
expect(requiresAdaptiveThinking(modelId)).toBe(expected);
},
);
```
**Context Used:** Do not attempt to comment on incorrect alphabetica... ([source](https://app.greptile.com/review/custom-context?memory=instruction-0))
How can I resolve this? If you propose a fix, please make it concise.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Problem
Users who pick the new claude-fable-5 model hit hard failures whenever a generation runs without a configured effort/thinking level — 8 users and 142 failed generations on 06-11 alone, all invisible to error monitoring (
$ai_http_status=None, so they never trip>= 400alerts).Since 06-10 these generations are deterministically rejected by Anthropic with
invalid_request_error: "thinking.type.disabled" is not supported for this model. The effort level only reaches the SDK when present, so a fable-5 session with no effort set lets the SDK/CLI default tothinking: { type: "disabled" }, which fable-5 (adaptive-only) rejects outright.Why: fable-5 was made user-selectable again by the 06-10 revert and only supports adaptive/enabled thinking — but our effort→options mapping leaves thinking disabled when no effort is configured, breaking ~2.6% of fable-5 generations.
Changes
requiresAdaptiveThinkingcapability flag insession/models.ts(alongside the existingMODELS_WITH_EFFORTset), withclaude-fable-5as the only current member.session/options.ts, default these models to a sharedDEFAULT_EFFORT("medium") when no effort is configured, so the SDK requeststhinking: { type: "adaptive" }+output_config: { effort }instead of disabled thinking. All other models keep their existing behavior (an explicit effort is still forwarded unchanged).modelIdintobuildSessionOptionsto gate the above.How did you test this?
models.test.ts(requiresAdaptiveThinkingmatches only fable-5) andoptions.test.ts(fable-5 defaults tomedium; explicit effort forwarded unchanged; opus-4-8 and no-model-id leave effort unset). Ranpnpm --filter @posthog/agenton both files — all pass.pnpm --filter @posthog/agent typecheckpasses; Biome clean; host-boundary check shows no new violations.Automatic notifications
Created with PostHog Code from an inbox report.