feat(agent): add in-pipeline pre-PR self-review phase#263
Open
nizar-lahlali wants to merge 2 commits into
Open
feat(agent): add in-pipeline pre-PR self-review phase#263nizar-lahlali wants to merge 2 commits into
nizar-lahlali wants to merge 2 commits into
Conversation
Adds an optional self-review phase between agent execution and post-hooks where the LLM critiques its own cumulative diff before the PR is created. This improves first-pass PR quality by catching bugs, style issues, and test gaps before human review. - New self_review.py orchestration module with run_self_review() - New prompts/self_review.py with focused review prompt template - TaskConfig extended with self_review_enabled and self_review_max_turns - Fields threaded through build_config, get_config, server, pipeline - Fail-open: self-review errors never block PR creation - Uses remaining turns/budget from original allocation (capped) - Feature is opt-in (disabled by default)
After the self-review phase completes, the agent writes a structured summary of findings to `.self-review-summary.md`. The pipeline reads this file, posts it as a PR comment via `gh pr comment`, then deletes it so it never appears in the PR diff. Fail-open: if the file is missing or the comment fails to post, the pipeline continues normally.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
self_review.pyorchestration module withrun_self_review()that generates the diff, invokesrun_agent()a second time with a review-focused prompt, and accumulates turns/costprompts/self_review.pywith a focused review prompt template (correctness, bugs, security, style, test gaps checklist)TaskConfigextended withself_review_enabled(defaultFalse) andself_review_max_turns(default5)build_config(),get_config(),server.py_extract_invocation_params(), andpipeline.pyrun_task().self-review-summary.mdwith structured findings, the pipeline posts it viagh pr comment, then deletes the file so it never appears in the diffDesign decisions:
max_turnsallocation (capped atself_review_max_turns)run_agent()call gives the model fresh context and a clean review perspectivepr_review(read-only), empty diff, no remaining turns/budgetCloses #262
Test plan
tests/test_self_review.py) — includes 13 new tests for summary reader and PR comment postingSELF_REVIEW_ENABLED=truein local batch mode to verify the phase executes and summary is postedself_review_enabled: truein the orchestrator payload