Conversational AI in Customer Service: From Hype to Reality
A short map out of the hype: the 90-day plan, what conversational AI does well, where it still falls short and how to measure without fooling yourself.

Marlos Carmo
Founder of Tolky
Updated on November 15, 2026

Share
This is the entry point. For the complete guide, see generative AI customer service for businesses; for the single-platform view (support, sales and relationship), see AI in customer service, support, sales and relationship.
"AI in customer service" stopped being a buzzword and became operational reality for companies that take CX seriously. But the gap between what vendors promise and what happens in operations is still large. This text is an honest map of that gap and how to cross it.
How to Start: The First 90 Days
Days 1 to 30: Baseline and Knowledge Base
Measure the starting point (response time, time to resolution, volume per channel, CSAT) and document the 30 most frequent questions with ideal answers. This base is the fuel of AI: garbage in, garbage out.
Days 31 to 60: Pilot Queue with Human Review
Turn on AI in one channel and one specific queue (WhatsApp is usually the best start because of volume), with intense human supervision. Every conversation should be reviewed in this phase.
Days 61 to 90: Calibration and Criteria to Scale
With real data, calibrate the human handoff rules, improve the knowledge base and set objective criteria to expand volume or channels: for example, quality of reviewed answers, handoff rate and satisfaction relative to the baseline.
For the pilot, Tolky's Individual plan is free; see the plans at /precos and the documentation at docs.tolky.to.
What Conversational AI Does Well
What It Masters
Smart triage and routing. A well-configured LLM identifies customer intent in seconds and routes to the right department with more precision than keyword rules.
Answers to frequent questions. In many operations, a good part of tickets are variations of a small set of questions. AI handles them with consistency and speed.
Context summaries for agents. When the customer reaches a human, the agent receives a briefing with history, sentiment and previous attempts, with no repetition for the customer.
What Is Still a Challenge
Complex emotional nuance. AI may answer in a technically correct but emotionally wrong way in a crisis. Smart handoff to a human remains the right answer in these cases.
Processes that require action in legacy systems. Integrating AI with old ERPs is still serious engineering work.

The Architecture of a Hybrid Operation
The model that works is not "AI replaces humans", but AI as first level and a specialized human as second level. The split below is illustrative; the real percentage depends on your operation.
Loading diagram
Configuring the AI to Human Transition
The decision of when to hand off to a human should consider:
- Negative sentiment above a defined threshold
- Attempts without resolution: if AI has not resolved it in 3 turns, escalate
- Escalation keywords: "cancel contract", "lawsuit", "consumer protection"
- Explicit customer request: always respect it
// Example escalation rule (starting values to calibrate)
const shouldEscalate = (context: ConversationContext) => {
return (
context.sentiment < -0.6 ||
context.unresolvedTurns >= 3 ||
context.hasEscalationKeyword ||
context.clientRequestedHuman
);
};Metrics That Matter
Do not accept "bot resolution rate" as your only metric, because it is easy to manipulate. Details on calculating return are in the article on AI automation ROI. As a starting editorial reference (not universal targets; calibrate against your baseline):
| Metric | What it measures | Editorial reference |
|---|---|---|
| Post-AI CSAT | Real satisfaction | Equal to or above baseline |
| Quality containment | % resolved without customer return | Rising, without CSAT drop |
| Time to resolution | First contact to closure | Varies by case type |
| Escalation rate | % that needed a human | Track the trend |
Conversational AI is not a set-it-and-forget-it button. It is an operation that requires continuous curation. Those who understood this are getting real results; those who bought the "10-minute setup" promise were disappointed.
Want to see how Tolky implements this architecture? Talk to our team or start with the plans.
Share
Tags
- #conversational ai
- #customer service
- 90-day plan
- ai pilot
- human handoff
About the author

Marlos Carmo
Founder of Tolky
Marlos Carmo is an AI entrepreneur and founder of Tolky, the conversational-era infrastructure and AI CRM that unifies intelligent service, multi-channel support (such as WhatsApp and voice), live CRM, and operational intelligence in a single ecosystem. He is a finalist for the SXSW Innovation Awards and a member of Francesco's Economy, a global network of young entrepreneurs focused on innovation and social impact. He works connecting Artificial Intelligence and digital transformation in projects for large organizations.
Next step
Want to bring this to your operation?
Tati, Tolky's AI assistant, understands your scenario in a few questions, suggests the best path and connects you with someone from the team, with full context.


