Kartik N V J K

I trust the 1,200-run result more because you also scored it with @shadcn/lint, which onsystem doesn't control. Going from 4.97 to 2.64 on an outside linter tells me more than the 0.00 from onsystem's own check. What's in the 2.64 that's left: rules onsystem doesn't cover yet, or code the agent wrote outside the files the hook checked?

Agents keep editing after they get a yes, so tying the approval to patch A and refusing it for patch B closes a gap I rarely see covered. I also liked the reminder that separate context windows aren't security boundaries, since most multi-agent diagrams I see draw them as if they were. You say to give each approval an expiration. For a first deployment, what do you start with, minutes or the life of the task?

Anyone who reviews AI output by eye should see your ACL example. Both versions look equally tidy, and only the order of two lines decides whether Telnet on port 23 gets through. I keep hitting the same thing with LLM evals, where a requirement nobody wrote down passes every check. In your workflow, would the engineer write "deny Telnet" as a Batfish policy check before asking the AI for a config, or pull it from the prompt afterwards?

Push time becoming the speed limit when an agent commits after almost every small step matches my own agent runs, which commit far more often than I ever did. 7.38 billion in a month is still a wild number. If secret scanning now runs alongside the write instead of before the push is acknowledged, does GitHub say what happens when a scan fails after the agent was already told the push went through?

Agents flagging a stale entry in only 4% of runs worries me more than any of the wins, because I'd expect a confidently wrong memory to spread faster than a fix. The 12% to 79% jump for Sonnet shows the channel works when the fact is in there. When a learned fix later turns out to be wrong and already has upvotes, can a few down-reactions push it out of the top five, or does someone have to delete it by hand?

"A reviewer that can fix what it finds is a second author" is going on a sticky note above my desk. The gap I keep watching for is the writer editing the tests in that same commit, since step 5 runs whatever tests the commit carries. Do you have a rule in RULES.md that makes a loosened or deleted assertion an automatic CHANGES REQUIRED?

test_counts_at_runtime is a nasty one. An ImportError dressed as a red test reads exactly like a real catch in a CI summary. I do a version of Check 1 for eval cases: a new case has to fail on the old prompt before I believe it guards anything. To your question, I'd run Check 1 on every agent PR and save hand-written mutants for diffs that touch a boundary like that range() bound.

I once watched a trace go green end to end on an agent that had done nothing, so the 200 OK table landed. Every retry wrapper I've written should have treated a timeout as unknown instead of failed. Who writes the expected-state contract in your setup? If the agent drafts it before the call, it can write a finish line it already knows it will cross.

Making the preview the real undo, rolled back, is the line I'm taking with me. A separate preview path would drift from the real one sooner or later. My worry is from the agent side: if someone undoes a run mid-session, the agent still believes its funnel exists. When its next write gets refused for joining an undone run, does the error say why, so it doesn't just open a fresh run and put everything back?

The 60 minutes shrinking to a 15 minute saving is the number I wish more agent comparisons showed, because the two-minute patch is the only part anyone screenshots. In my runs the repair minutes swing the most between tasks that look the same size. In the pilot you outline, would you report rework minutes per failure class, or one total next to the class counts?