fix: subscribeToBatch Zod stream crash on partial shape updates (#3510)#3699
fix: subscribeToBatch Zod stream crash on partial shape updates (#3510)#3699deepshekhardas wants to merge 1 commit into
Conversation
…gerdotdev#3510) Change zodShapeStream to skip unparseable chunks instead of crashing the entire stream. When ElectricSQL sends partial shape updates during initial sync (missing fields like taskIdentifier, friendlyId, createdAt), the Zod safeParse fails and controller.error() terminates the stream for all subscribers. Now logs a warning and skips the chunk instead.
|
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
WalkthroughThis PR modifies the error-handling logic in Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Hi @deepshekhardas, thanks for your interest in contributing! This project requires that pull request authors are vouched, and you are not in the list of vouched users. This PR will be closed automatically. See https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md for more details. |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| } else if (typeof console !== "undefined") { | ||
| console.warn( | ||
| `[zodShapeStream] Skipping unparseable shape chunk: ${result.error.message}` | ||
| ); | ||
| } |
There was a problem hiding this comment.
🟡 Parse failures silently dropped without notifying the onError callback
When a shape chunk fails Zod validation, the new code logs a console.warn and silently drops the chunk, but never invokes the options.onError callback (stream.ts:19). This means consumers have no programmatic way to detect that data is being silently discarded. In the worst case (e.g., a schema version mismatch between server and client after an upgrade), ALL chunks would fail to parse, causing the stream to produce zero data while appearing healthy — no error is thrown, no onError is called, and the consumer's for await loop hangs forever. For RunSubscription consumers (runStream.ts:616-637), this could mean a realtime subscription that never delivers any run updates and never completes, with no signal to the caller that something is wrong. The previous behavior (controller.error(...)) at least surfaced the problem immediately. The fix should invoke options?.onError?.(...) when parse failures occur, so consumers can detect and handle dropped data.
| } else if (typeof console !== "undefined") { | |
| console.warn( | |
| `[zodShapeStream] Skipping unparseable shape chunk: ${result.error.message}` | |
| ); | |
| } | |
| } else { | |
| if (typeof console !== "undefined") { | |
| console.warn( | |
| `[zodShapeStream] Skipping unparseable shape chunk: ${result.error.message}` | |
| ); | |
| } | |
| options?.onError?.(new Error(`Unable to parse shape: ${result.error.message}`)); | |
| } |
Was this helpful? React with 👍 or 👎 to provide feedback.
|
Hi team, this fixes #3510 — subscribeToBatch/subscribeToRun stream crashes when ElectricSQL sends partial shape updates during initial sync. Stream now skips unparseable chunks instead of terminating. Would appreciate a review and merge. Thanks! |
1 participant
When ElectricSQL sends partial shape updates during initial sync (where not all columns are present in every message), zodShapeStream would crash the entire ReadableStream by calling controller.error(). This killed the subscription for all downstream consumers.
Fixes the issue by logging a warning and skipping unparseable chunks instead of terminating the stream. The RowState in ReadableShapeStream correctly accumulates full row data across insert/update messages and only emits when up-to-date, but edge cases during reconnection or initial sync could emit incomplete rows.