Nice walkthrough. One area I'd pay particular attention to before using this in a production repository is the reliability of the review itself. A diff-based prompt is a useful starting point, but truncating a large diff at a fixed character limit can remove context that may be important for identifying cross-file impact. I'd consider adding a few safeguards: Repository-aware context retrieval instead of relying only on the PR diff. Structured response validation rather than relying on a broad regex to extract JSON. Clear separation between deterministic findings and AI-inferred suggestions. Actual test execution with distinct outcomes for passed, failed, timed out and not executed. Revalidation after a suggested fix is applied. Tracking dismissed findings to help identify recurring false positives. I also agree with keeping human approval in the workflow. An AI review result should provide evidence for the reviewer, not silently become a substitute for developer judgment. I've been exploring these problems while building a self-hosted Java/Spring Boot PR review tool. The distinction between generating review comments and validating their reliability is particularly important.
