I think the middle ground is to move the complexity away from the agent rather than trying to eliminate it entirely. For example, with a Secret Broker, I'd separate credential retrieval from command execution. The agent could request a specific secret through a narrow interface, while the broker validates the request, enforces access policies, and injects the credential directly into the process that needs it. The agent never gets access to the underlying vault token or unrestricted vault access. I'd also combine that with short-lived credentials and scoped permissions. GitHub Actions OIDC is a good example of this approach for deployment workflows. Another thing I'd be careful about is approval fatigue. If every operation triggers a popup, people eventually start approving things out of habit. I'd rather automate low-risk, reversible operations and reserve explicit approval for sensitive actions such as production deployments, external uploads, or changes involving privileged credentials. The goal isn't to make the agent completely trusted or completely restricted. It's to design the boundaries so that even if the agent makes a bad decision, the damage stays contained.
