`_remote_debugging`: unbound memory growth when recording binary and the thread is sleeping

[maurycy]

Bug report

Bug description:

When recording binary files, pending RLEs are flushed when the stack is changed:

cpython/Modules/_remote_debugging/binary_io_writer.c

Lines 1014 to 1018 in 29a920e

	/* Stack changed - flush any pending RLE first */
	if (entry->has_pending_rle) {
	if (flush_pending_rle(writer, entry) < 0) {
	return -1;
	}

or on finalize:

cpython/Modules/_remote_debugging/binary_io_writer.c

Lines 1077 to 1083 in 29a920e

	for (size_t i = 0; i < writer->thread_count; i++) {
	if (writer->thread_entries[i].has_pending_rle) {
	if (flush_pending_rle(writer, &writer->thread_entries[i]) < 0) {
	return -1;
	}
	}
	}

So, a sleeper can cause a memory leak:

cpython/Modules/_remote_debugging/binary_io_writer.c

Lines 996 to 1012 in 29a920e

	if (encoding == STACK_REPEAT) {
	/* Buffer this sample for RLE.
	*
	* STACK_REPEAT also covers the "first sample for a fresh thread,
	* empty stack" case: a new ThreadEntry has prev_stack_depth == 0
	* and a zero-initialized prev_stack, so compare_stacks() returns
	* STACK_REPEAT against an empty curr_stack (depth 0). Buffering
	* it here is correct; the RLE flush path emits it as a normal
	* STACK_REPEAT record. */
	if (GROW_ARRAY(entry->pending_rle, entry->pending_rle_count,
	entry->pending_rle_capacity, PendingRLESample) < 0) {
	return -1;
	}
	entry->pending_rle[entry->pending_rle_count].timestamp_delta = delta;
	entry->pending_rle[entry->pending_rle_count].status = status;
	entry->pending_rle_count++;
	entry->has_pending_rle = 1;

Reproduction

Another one found under "let's keep recording binary for a while and see what happens", but the very minimal repro is:

2026-06-11T20:13:39.406750000+0200 maurycy@gimel /Users/maurycy/work/cpython (main d2e27ac?) % ./python.exe -c "
import time
time.sleep(3600)
" & TARGET=$!

sudo ./python.exe -m profiling.sampling attach --binary -r 1000khz -d 300 -o /tmp/rle.bin $TARGET &
sleep 2; PROF=$(pgrep -fn "profiling.sampling attach")
for i in $(seq 5); do printf "t=%2dmin  RSS=%d MB\n" $i $(($(ps -o rss= -p $PROF|tr -d ' ')/1024)); sleep 60; done
[1] 62736
[2] 62737
t= 1min  RSS=46 MB
t= 2min  RSS=348 MB
t= 3min  RSS=645 MB
t= 4min  RSS=945 MB
t= 5min  RSS=1246 MB
Captured 98,200,648 samples in 300.00 seconds
Sample rate: 327,335.49 samples/sec
Error rate: 0.00
  Binary Encoding:
    Records:          1
      RLE repeat:     0 (0.0%) [0 samples]
      Full stack:     1 (100.0%)
      Suffix match:   0 (0.0%)
      Pop-push:       0 (0.0%)
  Frame Efficiency:
    Frames written:   1
    Frames saved:     0 (0.0%)
    Bytes (pre-zstd): 18 B
Warning: missed 201799353 samples from the expected total of 300000001 (67.27%)
Binary profile written to /tmp/rle.bin (98200648 samples)
[2]  + done       sudo ./python.exe -m profiling.sampling attach --binary -r 1000khz -d 300 -o 
2026-06-11T20:18:42.018130000+0200 maurycy@gimel /Users/maurycy/work/cpython (main d2e27ac?) % 

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

[maurycy]

cc @pablogsal I will take this one, it's making my life under mach_vm_remap testing hard :-)