PracticeWorking with AI: Delegation & Oversight
What kind of pilot?
The demo and the deployment
The situation
A vendor has demonstrated a support agent that replies to customers directly. Their claim: it handles 80% of tier-1 tickets autonomously. Noor has pulled your own tier-1 ticket distribution to set beside it.
Elias · Vendor account executive: “Most customers go live within a week. Shall I set the pilot up?”
The demo was genuinely impressive, and the 80% may well be true for someone. You have one question before you answer.
What you can see
E01 · The claim
'Handles 80% of tier-1 support tickets autonomously.' Stated in the vendor deck and repeated in the demo. No definition of 'handles' or of the customer base it was measured on is given in the deck.
E02 · The demo
Three tickets resolved live in under a minute each: a password reset, a request for an invoice copy, and a plan downgrade. The replies were accurate and well written.
E03 · Your tier-1 mix
About 1,900 tier-1 tickets a month. Three categories make up 58%: accounts suspended for non-payment, seat reassignment after someone leaves a customer's company, and accounts mid-migration between plans. Each needs the agent to read account state and, usually, change it.
Step 1 of 2
What do you ask?
Your first choice is kept. Changing your mind later counts as a retry.