# AI voice agent for customer service: what it is and how it works

> What an AI voice agent for customer service is, how it listens, understands and answers, how it differs from a traditional IVR, where to use it, what it costs and what to watch out for.

- **HTML version:** https://tolky.to/en/blog/ai-voice-agent-customer-service
- **Author:** [Marlos Carmo](https://tolky.to/en/author/marlos), Co-founder of Tolky
- **Published:** November 19, 2026
- **Reviewed:** November 19, 2026
- **Language:** en

## Summary

An **AI voice agent** answers or places calls in natural language: it transcribes what the customer says, decides the reply with a language model and speaks back with a synthetic voice, handing off to a person when the case calls for it. It replaces the keypad IVR menu with a conversation and works on the website, on WhatsApp calls and on the phone.

Search for "voice AI" and you get two things mixed together: tools that clone or generate voices for videos, and agents that answer customer calls. This guide covers only the second. The question here is how a company puts an AI on the phone, what it resolves on its own and where a person needs to step in.

Below: how a voice agent works, how it differs from an IVR, the main use cases, the channels, the cost, the legal care and the metrics.

*Updated in November 2026.*

## What is an AI voice agent?

**An AI voice agent is software that answers or places calls, understands what the person says in natural language, replies with a synthetic voice and completes tasks during the call.** It is also called a [voice agent](https://tolky.to/en/glossary/ai-voice-agent), voicebot or conversational IVR.

The difference from a keypad IVR is that the agent holds a conversation. The customer says "I need to move my Thursday appointment" and the agent understands the request, checks the calendar and offers a new time, without making anyone press 1, 2 or 3.

In this guide, "voice AI" does not mean voice cloning or narration for videos. It means customer service: a call received or placed by the company, with a customer on the line and an outcome to deliver.

## How does an AI voice agent work?

**An AI voice agent works in three steps that repeat with every turn: transcribe the audio into text, decide the reply with a language model and turn the reply back into speech.** All of it has to happen in a moment for the conversation to sound natural.

1. **Speech to text ([STT](https://tolky.to/en/glossary/stt)).** Speech recognition transcribes what the customer says in real time, including accents and background noise.
2. **Language model (LLM).** The model reads the transcript, checks the knowledge base and company systems, and decides what to say or do.
3. **Text to speech ([TTS](https://tolky.to/en/glossary/tts)).** Speech synthesis turns the reply into audio, in the chosen voice and accent.

Two factors separate a good voice agent from a bad one. The first is [latency](https://tolky.to/en/glossary/latency), the time between the customer finishing a sentence and hearing the answer. Long pauses make people repeat themselves or think the call dropped, so these systems stream every step and favor short replies.

The second is [barge-in](https://tolky.to/en/glossary/barge-in), the ability to stop talking when the customer interrupts. Without it, callers must hear the whole sentence even when they already know what they want.

There are two common architectures. In **cascade mode**, the three steps run in sequence, which makes each one easier to control. In **real-time mode**, a single model listens and speaks at the same time, which makes the conversation more fluid.

## Voice agent, traditional IVR and semantic IVR: what is the difference?

**A traditional IVR routes the call through keypad menus; a semantic IVR understands free speech to route or resolve; an AI voice agent holds the conversation end to end and completes tasks.** In practice, the semantic IVR is the front door of a voice agent, and many companies combine both ideas.

| Criterion | Traditional IVR | Semantic IVR | AI voice agent |
| :--- | :--- | :--- | :--- |
| How the customer asks | Keypad and fixed options | Free speech, interpreted | Free speech, in conversation |
| Changing the subject | Back to the main menu | Reroutes | The conversation continues |
| Interrupting | Keypad only | Depends on the system | Barge-in on speech |
| Checking systems | Routes to a queue | Can check | Checks and answers |
| Resolution without a person | Very simple requests | Frequent requests | Frequent requests and tasks with clear rules |
| Handoff | No context | With the detected intent | With the reason and the transcript |

The [semantic IVR](https://tolky.to/en/glossary/semantic-ivr) has its own post with end-to-end examples: [voice AI and semantic IVR](https://tolky.to/en/blog/voice-ai-semantic-ivr). Here, the focus is the voice agent as a whole, including outbound calls.

## What is an AI voice agent used for?

**A voice agent handles repetitive inbound calls without a queue and places calls at scale with a script, always with a way out to a person.** The most common uses fall into four groups:

- **Inbound service.** Duplicate invoices, order status, opening hours, triage of the reason for the call. The AI resolves the repetitive part and transfers the rest to the [contact center](https://tolky.to/en/blog/ai-powered-contact-center).
- **Outbound calls.** First contact with leads, customer reactivation, renewal invitations. At Tolky these are [voice campaigns](https://tolky.to/en/solutions/voice-campaigns).
- **Confirmations and scheduling.** Confirm, reschedule or cancel appointments and visits, common in [healthcare](https://tolky.to/en/industries/healthcare) and any business that books time slots.
- **Collections.** Remind customers of a due date, state the amount and offer the payment channel, with a handoff when the customer wants to negotiate. See the [collections](https://tolky.to/en/departments/collections) area.

For long, sensitive calls, such as serious complaints or complex negotiations, the best use is to triage and hand off to a person with the context ready.

## Which channels can a voice agent answer?

**A voice agent can answer on three channels: voice on the website, in the browser; WhatsApp calls, through Meta's Calling API; and regular phone lines, through SIP telephony.** Each one has its own rules.

- **Web.** The visitor clicks and talks straight from the browser, with nothing to install. Good for support and pre-sales on your own site.
- **WhatsApp.** The [Calling API](https://tolky.to/en/glossary/whatsapp-voice-calls) lets a business receive and place voice calls inside WhatsApp. According to [Meta's documentation](https://developers.facebook.com/documentation/business-messaging/whatsapp/calling), checked on November 19, 2026, calling a business is free for the user. For the business to call, the contact must grant permission first.
- **Phone.** The agent answers a regular number, with [SIP telephony](https://tolky.to/en/glossary/sip-telephony) connecting the phone network to the platform. It is the channel that reaches anyone, including people who do not use WhatsApp.

On all three, the transcript of each call needs to land in the customer's history, next to the text conversations.

## How much does AI voice customer service cost?

**The cost of AI voice customer service adds up three parts: the channel fee (phone or WhatsApp), voice and AI processing per minute, and the platform that orchestrates it all.** Since each vendor bundles these parts differently, compare by cost per resolved call, not by the per-minute price alone.

On WhatsApp, Meta publishes the rules on the [Calling API pricing page](https://developers.facebook.com/documentation/business-messaging/whatsapp/calling/pricing), checked on November 19, 2026:

- User-initiated calls are free.
- Business-initiated calls are charged by duration, in 6-second pulses, based on the country of the number called and the monthly volume tier.
- The call permission request is a message and follows regular per-message pricing.

On phone lines, the per-minute rate depends on the carrier and the destination (landline or mobile). Voice processing weighs more on long calls, so short replies and a good script cut both cost and latency. To estimate the return, use the [customer service ROI calculator](https://tolky.to/en/resources/customer-service-roi-calculator).

## What should you watch out for with voice AI?

**Voice AI calls for four precautions: disclose that it is an AI, guarantee a handoff to a person, treat audio and transcripts as personal data, and follow your sector's rules on recordings and wait times.** They are what keep complaints and legal risk down.

- **Transparency.** Say at the start that the customer is talking to a virtual assistant. Hiding it tends to backfire when the person notices.
- **Handoff to a human.** Many markets regulate this. In Brazil, for example, [Decree 13,135/2026](https://www.planalto.gov.br/ccivil_03/_Ato2023-2026/2026/decreto/d13135.htm), which governs customer service for federally regulated services and digital financial services, allows artificial intelligence for triage and resolution as long as the consumer can get real-time service from a person. It also requires human service from 8 a.m. to 8 p.m. and caps service at five levels (checked on November 19, 2026). Set your rules with the [AI-to-human handoff guide](https://tolky.to/en/blog/ai-to-human-handoff-7-rules).
- **Data protection.** Voice and transcripts are personal data. Brazil's [LGPD](https://tolky.to/en/glossary/lgpd-in-customer-service) ([Law 13,709/2018](https://www.planalto.gov.br/ccivil_03/_ato2015-2018/2018/lei/l13709.htm)), like the GDPR, requires a stated purpose, minimal use and transparency with the data subject (art. 6).
- **Recording and records.** The same Brazilian decree requires keeping call recordings for at least 90 days (art. 19). Check your sector's rule before deciding where and for how long audio is kept.

## How do you measure whether the voice agent is working?

**Measure a voice agent by what it resolves and by the quality of its handoffs, not by the number of calls answered.** Answering a lot and resolving little just pushes the queue to later.

| Metric | What it shows | Warning sign |
| :--- | :--- | :--- |
| AI resolution | Calls closed without a person, with the request handled | High, but with many repeat calls the next day |
| Transfer rate | How much goes to the team and why | Empty reasons, or repeated reasons with no fix in the knowledge base |
| Abandonment | Callers who hang up before being served or resolved | Spikes after script changes |
| Average handle time | Call duration | Long calls for simple requests |
| First contact resolution | Cases solved without a second call | Customers calling back for the same reason |
| Post-call NPS | How callers feel | Drops on calls handled only by the AI |

To calculate each one, see [how to measure AI resolution rate](https://tolky.to/en/blog/ai-resolution-rate-how-to-measure) and the terms [FCR](https://tolky.to/en/glossary/first-contact-resolution) and [AHT](https://tolky.to/en/glossary/aht). Listen to a sample of calls every week: the transcript shows where the AI got it wrong or took too long.

## How Tolky handles voice

**Tolky is an AI conversational CRM where the same AI agent that answers by text also answers by voice: on the website, on WhatsApp calls and on the phone with a semantic IVR.** Telephony is enabled on activation, with the number and agent you choose. The features are on the [AI voice agent](https://tolky.to/en/features/ai-voice-agent) and [voice customer service](https://tolky.to/en/solutions/voice-customer-service) pages.

- **Configurable voice.** You pick the voice with a preview, the regional accent, the greeting and the reply length, in cascade or real-time mode.
- **Natural conversation.** The customer can interrupt the AI, and the [real-time transcript](https://tolky.to/en/glossary/real-time-transcription) follows the caller's language.
- **People on the line.** The team follows live calls and can listen in. When an operator joins the audio, the AI goes silent; if the owner does not pick up, the call rings other operators.
- **One history.** The transcript stays in the contact's history, next to the text conversations.
- **WhatsApp with permission.** To call over WhatsApp, Tolky sends the permission request and shows its status in the conversation.
- **Voice campaigns.** The AI calls your base over WhatsApp or telephony, with a script, a goal, time windows and a daily spending cap. Every call opens with a transcription notice that cannot be turned off.

![Real screenshot of Tolky's Identity screen, in dark theme, where you set the AI agent's role, the opening message, the Sales and Commercial Relationship mission and the agent's behavior](https://tolky.to/screenshots/identidade.png)

**What Tolky does not do:** it is not a narration or voice cloning tool for videos, and telephony is not on by default, because it requires activation. If your operation must keep audio recordings for a set period, confirm with the team how that works in your case.

## Frequently asked questions

### What is an AI voice agent?

It is an assistant that answers or places calls in natural language. It transcribes what the customer says, decides the reply with a language model and responds with a synthetic voice, handing off to a person when needed.

### What is the difference between a conversational IVR and a traditional IVR?

A traditional IVR uses keypad menus and fixed options. A conversational, or semantic, IVR understands what the customer says in their own words and routes or resolves the request without "press 1".

### Can an AI voice agent answer WhatsApp calls?

Yes. Meta's Calling API lets businesses receive and place voice calls on WhatsApp, and those calls can be handled by a voice agent. For the business to call, the contact must grant permission first.

### Does the customer need to know they are talking to an AI?

It is the recommended practice: disclosing it at the start prevents distrust and complaints. Some regulations, such as Brazil's customer service rules, also require that consumers can reach a person.

### Does voice AI replace the contact center team?

No. It absorbs repetitive calls and first-level triage, and the team keeps the cases that need judgment. A person must be able to join the call at any time.

### How much does an AI voice agent cost?

It depends on the channel, call length and platform. On WhatsApp, customer-initiated calls are free and business-initiated calls are charged by duration; on phone lines, there is the carrier fee. Compare by cost per resolved call.

---

## Conclusion

An AI voice agent puts a conversation where the keypad menu used to be: the customer talks, the AI understands, resolves what has clear rules and hands the rest to a person with the transcript ready. Start with the most repetitive calls, disclose that it is an AI, guarantee a way out to the team and measure by resolution.

[Explore the voice agent](https://tolky.to/en/features/ai-voice-agent), [see voice campaigns](https://tolky.to/en/solutions/voice-campaigns), [see the plans](https://tolky.to/en/pricing) or [talk to the team →](https://tolky.to/en/contact). Let's talk?

---

This is the Markdown version of the page. The full version, with images and navigation, is at https://tolky.to/en/blog/ai-voice-agent-customer-service

Tolky: https://tolky.to
