Skip to content

gh-148223: Avoid unnecessary NotShareableError in XIData FULL_FALLBACK#148224

Open
jrfk wants to merge 1 commit intopython:mainfrom
jrfk:xidata-skip-exc-on-fallback
Open

gh-148223: Avoid unnecessary NotShareableError in XIData FULL_FALLBACK#148224
jrfk wants to merge 1 commit intopython:mainfrom
jrfk:xidata-skip-exc-on-fallback

Conversation

@jrfk
Copy link
Copy Markdown
Contributor

@jrfk jrfk commented Apr 7, 2026

Summary

Avoid unnecessary NotShareableError creation in _PyObject_GetXIData() on the FULL_FALLBACK path.

For types not present in the XIData registry, the previous code always called _get_xidata(), which raised NotShareableError before falling back to function/pickle handling. When pickle fallback succeeded, that exception was immediately discarded. This change adds a registry pre-check so unregistered types can skip _get_xidata() and go straight to the fallback path.

Follow-up to gh-148072 / GH-148125 (pickle cache).

Changes

  • restructure the FULL_FALLBACK branch in Python/crossinterp.c
  • probe the XIData registry with lookup_getdata() before calling _get_xidata()
  • keep the existing path for registered types
  • skip unnecessary exception creation for unregistered types
  • preserve the previous NotShareableError behavior on total failure

Behavior

This is intended as a performance-only change.

  • registered types still follow the existing path
  • unregistered types use the same fallback chain as before
  • on failure, the same final NotShareableError is raised as in the non-optimized path

Benchmarks

Baseline is upstream/main (after GH-148125). identity(x) round-trip, max_workers=1, Apple M2.

Data Before After Speedup
list(100) 0.14ms 0.06ms 2.3x
dict(100) 0.20ms 0.08ms 2.5x
list(10000) 3.45ms 0.53ms 6.5x
dict(10000) 10.66ms 2.52ms 4.2x

Test plan

  • ./python -m test test_interpreters — 172 tests passed
  • ./python -m test test_concurrent_futures — 380 tests passed (including test_interpreter_pool)

…ALLBACK

Skip _get_xidata() for types not in the XIData registry when using
the FULL_FALLBACK path in _PyObject_GetXIData().  This avoids creating
and discarding a NotShareableError exception on every successful pickle
fallback transfer.

Registered types (None, bool, int, float, str, bytes, tuple) still
follow the existing path unchanged.  On total failure, the same
NotShareableError is raised as before.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant