Customer Service AI Chatbot Readiness Checklist

TL;DR

A support bot is not ready because it can answer common questions. It is ready when it knows what it is allowed to say, when it must stop and hand the customer to a person, and how you will catch it being wrong before the customer does. The most common reason bots fail is not the model. It is the goal they are given: "close as many tickets as possible without a human." That goal rewards the bot for ending conversations, and a confident wrong answer ends a conversation just as well as a correct one. Fix the goal first. Then ship the bot in a narrow lane with hard stop rules, and measure it with a number that cannot be improved by a wrong answer.

The mistake behind most failed support bots

The most common way a support bot fails has nothing to do with how good the AI is.

It was launched to take work off the team: handle as much as possible without a person. Whatever number gets put on the dashboard to track that becomes the thing the bot is optimized for. Here is the problem in one sentence: a wrong answer ends a conversation just as efficiently as a right one. So a bot graded on "did we avoid involving a person" will look like it is winning while it quietly sends customers away with bad answers.

This is the sentence a chatbot vendor will argue with, so state it plainly: the loops, the bad handoffs, the technically-correct-but-useless replies are not four separate problems to tune out one at a time. They are one problem, the wrong goal, showing up in four places. You cannot prompt-engineer your way out of an incentive.

That claim is testable, which is the point. Stand a bot up against "tickets handled without a person," watch that number rise, then watch what share of those same tickets come back within a week. If reopened tickets do not fall while "handled without a person" rises, the bot resolved nothing. It moved the cost downstream, and it tends to surface fast — often by the next billing cycle or reporting period — when the same customers return, angrier, and an agent now has to undo the conversation before they can begin.

That last part is the real cost, and it is worth being concrete: a ticket the bot has already mishandled is more expensive to resolve than the original ticket would have been. The agent has to read what happened, calm the customer, walk back whatever the bot implied, and only then solve the actual problem. A bot built to avoid people can therefore raise your total cost to serve while reporting that it is succeeding.

The one test you can run before launch

You do not need a pilot to see this coming. Ask one question:

Can the bot's main success number be improved by giving a customer a wrong answer?

If yes, you have not built a support system. You have built something graded on getting rid of people, and it will do exactly that. Change the number before you change the prompt. The rest of this briefing is how.

"It can answer common questions" is the wrong readiness bar

Answering common questions is the easy 80% and the part every demo shows you. Readiness is decided entirely by the other 20%: what happens when the bot is unsure, when the source material is thin or stale, when money or a promise is involved, or when the customer is already upset. A bot that is excellent at the easy 80% and undefined on the hard 20% is not 80% ready. It is not ready, because the 20% is where the brand damage and the cost live.

So the readiness questions are not "what can it answer." In priority order they are:

When must it stop and get a person? First, because it is the only question that contains the downside. Get this wrong and nothing else matters.
What is it allowed to say, and from which source? A bot with no defined source will confidently repeat whatever it was last fed, including the stale article.
What must it never say or promise? Money, eligibility, timing, policy exceptions.
What does the person who takes over receive? A handoff with no context is a new failure, not a save.
Who reads what it actually said, and how often? Unreviewed automation is not deployed; it is unsupervised.
What number proves it helped, and can that number be faked by a wrong answer?

Note the weighting. Five of the six are about the boundary, not the answers. That is the asymmetry teams get backwards.

The escalation rule set (use this one)

These are not principles. They are rules with thresholds you can hand to whoever configures the bot. It stops and routes to a person the moment any one is true:

The customer asks for a person, in any words. "Human," "agent," "representative," "someone real," "manager." This is a first-class, zero-friction exit, not a last resort the bot argues against. Zendesk's 2025 CX Trends research is the load-bearing point here: 95% of consumers say they want to know why an AI made the decision it did, and 80% of CX leaders now say transparency is non-negotiable for customer-facing AI — yet only 37% offer any reasoning today. A bot that hides what it is, or fights the customer trying to get past it, is failing the exact thing the research says customers now require. That's not a UX annoyance; it breaks the trust the rest of the system depends on, and it puts the relationship at risk, not just the ticket.
The same question comes back twice in one conversation. Repetition is a stop signal, not a retry signal. Two attempts at the same intent, then a person. No third loop.
The bot cannot ground its answer in an approved source. No current approved document behind the answer means the bot does not improvise. It says it is getting a person, and does.
The request touches money, eligibility, cancellation, a policy exception, a legal matter, or an account change. Escalates before the first reply, not after the customer asks twice. The bot may gather information; it may not answer.
Sentiment turns negative, or the customer arrives angry. Frustration plus an automated reply is the exact combination that produces the screenshots.
The account is high-value or contract-sensitive. For these, the bot runs in draft-only mode regardless of topic: it writes, a person sends.
It is a second contact on the same issue within about a week. The bot already failed this customer once. It does not get to try again on its own.

The handoff itself is a rule, not a hope. The person taking over receives, in one place: the full transcript, what the customer is actually trying to do, what the bot already told them, and what it could not confirm. If the customer has to re-explain anything, the handoff failed and you spent trust you did not need to spend.

Ship the prep, not the bot, first

The lowest-risk, highest-return first move is not a customer-facing bot at all. It is the same AI doing the support work behind the agent, where a person still sends every word the customer sees:

Summarize long ticket threads so triage is fast.
Draft replies the agent edits and approves.
Route and prioritize by topic and urgency.
Flag anger, money, and high-value accounts for fast human attention.
Cluster repeated complaints so leadership sees a pattern, not ten anecdotes.

This is not a consolation prize. It is how you build the approved-source library, the escalation logic, and the review habit a customer-facing bot would otherwise have to learn in public, on your customers. It is also exactly the sequence McKinsey describes for generative AI in customer assistance: simple, small, manageable steps first, then customer-facing use with a human in the loop, and only then more automation once leaders are genuinely confident. The reason isn't caution for its own sake — it's that the failure mode is irreversible exposure. Mistakes have to stay internal while the system is still wrong; a public bot makes them permanent and customer-facing instead. Earn the customer-facing lane; do not assume it.

What the bot must never do

A person owns any reply that commits the company, moves money, changes an account, or interprets policy. The bot may draft these; the customer does not receive them until a named person approves. The bot never invents policy, never promises timing the business has not guaranteed, and never confirms a refund, credit, or eligibility. These are not edge cases for later. They are the reason the review owner exists.

Measure it so the number cannot lie

Track one primary number and one that keeps it honest:

Primary: resolved and stayed resolved. The share of bot-handled contacts that did not come back within about a week. A wrong answer cannot inflate this, because a wrong answer comes back.
Honesty check: came back and got reassigned. If "handled without a person" rises while "came back within a week" does not fall, stop. The bot is not resolving; it is relocating cost onto your agents and your customers.

Also watch time to a real resolution, how often the escalation decision itself was right, and how heavy the review burden is. If you remember one sentence: a bot that ends conversations is not the same as a bot that resolves them, and only the reopen number knows the difference.

What not to do

Do not give the bot a goal that improves when a customer gets a wrong answer.
Do not point it at the whole knowledge base on day one.
Do not let it confirm money, eligibility, or policy.
Do not make the customer argue with it to reach a person.
Do not go customer-facing before the escalation rules and a named review owner exist.

Related workflow pages

Related field reports

Where to go next

The AI customer service chatbot implementation page explains how ADA scopes a support deployment behind the agent before it ever faces a customer, and the AI readiness assessment helps identify whether your sources, escalation rules, and review owner exist yet. To pressure-test the escalation rule set against your own tickets, request an implementation review.

FAQ

Is a support chatbot ready when it can answer common questions?

No. Common questions are the easy part every demo shows. Readiness is decided by what the bot does when it is unsure, when money or a promise is involved, or when the customer is upset. That is where the cost is.

What is the single biggest mistake teams make with support bots?

Giving the bot a goal that improves when it avoids a person. A wrong answer avoids a person just as well as a right one, so that goal quietly rewards bad answers. Fix the goal before the prompt.

What should a support chatbot never do?

Confirm refunds, credits, eligibility, or policy exceptions, or make timing and scope promises the business has not guaranteed. It can draft these; a person approves before the customer sees them.

How do we know if the bot is actually working?

Watch the share of bot-handled contacts that come back within about a week. If that does not fall while "handled without a person" rises, the bot is moving cost, not removing it.

What should we launch first?

The AI working behind the agent: summaries, drafted replies, routing, escalation flags, where a person still sends every customer-facing word. It builds the sources and rules a customer-facing bot needs without practicing on customers.