The part that usually decides whether this works is the data feeding the agent. Before letting it draft follow-ups or update CRM records, pick one narrow task, like a post-call follow-up, and give it a fixed template: the call notes, the prospect's role, the one next step agreed, and a hard rule that it can only reference facts from those notes. That stops invented details, which is the pitfall that burns trust fastest.
Second, keep a human approving anything that leaves the building for the first few weeks, and track edit rate. If reps rewrite more than a third of drafts, fix the prompt or inputs before widening scope.
Third, write CRM updates as suggestions in a notes field first, then promote to real fields once accuracy looks solid.