Fix: Extended Query Protocol Parse packets corrupted by interceptor#18
Draft
nik-localstack wants to merge 1 commit into
Draft
Fix: Extended Query Protocol Parse packets corrupted by interceptor#18nik-localstack wants to merge 1 commit into
nik-localstack wants to merge 1 commit into
Conversation
…y OID suffix The Parse packet body (type 'P') has the format: statement_name\x00 + query\x00 + int16(param_count) + uint32[] OIDs The old handler used data[1:-2] / data[-2:] which treated only the last 2 bytes as the param-count field, leaking binary OID bytes into the query slice fed to _intercept_query(). When a parameter type OID contains a byte >= 0x80 (e.g. jsonb OID 3802 = 0x00000EDA), the subsequent UTF-8 decode raised UnicodeDecodeError, causing the connection to drop or hang. Fix: use find(b'\x00') to locate the true boundaries of statement name and query text, so the binary suffix is never touched by the text decoder. Adds a regression test using psycopg v3 (Extended Query Protocol) with a jsonb+text parameterized INSERT through the proxy. Co-authored-by: GitHub Copilot <copilot@github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
psycopg v3 (and asyncpg) use the Extended Query Protocol, sending a Parse packet whose body ends with binary uint32 parameter-type OIDs.
The interceptor sliced the body with
data[1:-2], leaking those OID bytes into a UTF-8 decode.The jsonb OID (
3802=0x0EDA) contains0xDAwhich is invalid UTF-8 and as a result crashing the interceptor and leaving the connection in a broken state.In LocalStack this surfaced as an indefinite hang.
Fix
Use
find(b"\x00")to locate the true null-terminators of the statement name and query text, so the binary OID suffix is never fed to the text decoder.Tests
A new e2e regression test that connects via psycopg v3 through the proxy and executes a parameterized INSERT with jsonb and text arguments.