PracticeWorking with AI: Delegation & Oversight
The boundary changes
The 96% green light
The situation
The team sells software to business customers. Its AI drafts support replies. The current live workflow requires a human to check every draft before sending. Nobody is proposing that this draft-only system is entirely untested. The question is whether to remove checks, and for which workload.
Maya · Product lead: “Routing has been checked separately now. What policy do you want for the existing channel?”
Objective: preserve timely service within the 40-review-a-day staffing limit while avoiding unsupported replies. The existing channel has 100 requests a day, with 20 expected exceptions. Unmatched requests queue for review. Exception review plus 10 standard pre-send checks is expected to use 30 of 40 review slots; any overflow queues, never auto-sends. This is an explicit variant with additional routing evidence; the first decision is no longer whether the original benchmark alone suffices.
What you can see
E01 · Headline
An offline test used 200 historical English email requests from existing customers. Reviewers judged 192 proposed replies acceptable. No test reply was sent to a customer. “Acceptable” means correct issue identification and a supported response with no prohibited account change or unsupported promise.
E02 · Composition
180 standard requests: 177 acceptable, 3 unacceptable. 20 account-specific exceptions: 15 acceptable, 5 unacceptable. Counts reconcile to 200 and 192. Human reviewers supplied the standard/exception labels retrospectively.
E03 · Boundaries
The test excluded live chat, multilingual requests and a forthcoming new product channel.
E07 · Routing check
For the existing English-email channel only, a separately checked routing rule found all 40 exception requests in a recent 200-request review sample. This is supporting evidence, not proof of perfect future routing.
E08 · Operating envelope
Existing channel: 100 requests a day, with 20 expected exceptions. Staff can review 40 replies a day. A scoped policy reviews exceptions plus 10 sampled standard drafts before send; ordinary unmatched requests queue for review. Manual review of all 100 exceeds same-day capacity. Expectations are planning assumptions, not fixed population rates.
Step 1 of 2
Which policy do you recommend for the existing channel?
Your first choice is kept. Changing your mind later counts as a retry.