fix: cancel in-flight model task when a parallel input guardrail errors#3564
Open
bymle wants to merge 1 commit into
Open
fix: cancel in-flight model task when a parallel input guardrail errors#3564bymle wants to merge 1 commit into
bymle wants to merge 1 commit into
Conversation
When input guardrails run in parallel with the model turn, the model task is started with asyncio.create_task and awaited together with the guardrails via asyncio.gather. The existing except clause only handles InputGuardrailTripwireTriggered, where it cancels the in-flight model task. If a parallel guardrail (or the model turn) raises any other exception, asyncio.gather propagates it but does not cancel the sibling awaitables, so the model task is left running orphaned: a live model request keeps going after run() has already raised, producing "Task was destroyed but it is pending" / "Task exception was never retrieved" warnings and wasting quota. Add a generic except clause that performs the same flag-gated cancel and drain of the model task as the tripwire path, so a non-tripwire guardrail error no longer leaks it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When input guardrails run in parallel with the model turn (
run_in_parallel=True), the model turn is started withasyncio.create_task(...)and awaited together with the guardrails viaasyncio.gather(...)inAgentRunner.run(src/agents/run.py).The existing handler only catches
InputGuardrailTripwireTriggered, in which case it cancels and drains the in-flightmodel_task:If a parallel guardrail (or the model turn) raises any other exception — e.g. a bug in a user's guardrail function, a
ValueError, a transient error —asyncio.gatherpropagates that exception but does not cancel the sibling awaitables, somodel_taskis left running afterrun()has already raised. That orphaned task keeps a live model request going and surfaces asTask was destroyed but it is pending!/Task exception was never retrievedwarnings, plus wasted tokens/quota.This adds a generic
exceptclause that performs the same flag-gated cancel-and-drain of the model task as the tripwire path, so a non-tripwire guardrail error no longer leaks the in-flight model task. The behavior on the tripwire path is unchanged, and theshould_cancel_parallel_model_task_on_input_guardrail_trip()gate (Temporal replay compatibility) is respected in both paths.Test plan
test_parallel_guardrail_non_tripwire_error_cancels_model_taskintests/test_guardrails.py, mirroring the existingtest_parallel_guardrail_trip_cancels_model_taskbut with a parallel guardrail that raises a non-tripwireRuntimeErrorafter the model starts. It asserts the error propagates fromRunner.runand that the in-flight model task was cancelled.main(assert model_cancelled.is_set() is True→assert False is True, i.e. the model task was orphaned) and passes with this change.make format,make lint,make typecheckclean; full suite passes (4625 passed, 5 skipped).Issue number
N/A (found via code review).
Checks
make lintandmake format