Skip to content

4 Million AI Messages a Month: Why Relational AI Needs Infrastructure

We process 4 million AI messages every month. Behind that number is an engineering decision: treat conversational AI as critical infrastructure, robust and observable.

Marlos Carmo

Founder of Tolky

Updated on November 15, 2026

9 min read
4 Million AI Messages a Month: Why Relational AI Needs Infrastructure

Share

We crossed the mark of 4 million AI messages processed per month (Tolky internal data, measured in June 2026, when this milestone was first published). These are real conversations, with real customers, in operations that can't stop: support, sales, billing, service and relationships running in natural language, day and night, in companies whose revenue depends on it.

It's a number we're proud to celebrate. But internally it means something else: 4 million opportunities a month to break someone's trust. Every message is a customer waiting for an answer, a payment at stake, a complaint that could become a lawsuit, a lead deciding to buy or walk away. In conversational AI, volume isn't a trophy: it's a responsibility.

And here is the thesis of this article, the same one that guides our engineering: relational AI only works at scale if there is real infrastructure underneath it. A good model and a clever prompt aren't enough. What sustains millions of conversations with quality is what no one sees. That's exactly what separates a pilot that dazzles in the demo from an operation that holds up all year long.

What "4 million messages" actually demands

When a company decides to put AI at the center of its relationships, it isn't asking for a chatbot. It's asking the conversation to become a business-critical channel, on the same level as the ERP, the payment gateway or the banking core. And a critical channel has non-negotiable requirements:

  • Availability. If the AI goes down on Black Friday, on a month-end billing run, or during a PR crisis, that isn't "a bug": it's lost revenue and furious customers in real time.
  • Stable latency under load. Responding quickly at low volume is easy. Holding that speed when volume multiplies several times over during a campaign (a marketing push or a billing run, for example) is an architecture problem, not a model problem.
  • Consistency. The same question can't get a safe answer today and a hallucination tomorrow because something in the chain silently degraded.
  • Data isolation. Each customer operates on its own data, with no leakage across accounts, and a trail of who saw and changed what.
  • Recovery. When something fails (and at scale, something always fails), the system has to degrade gracefully and recover on its own, without taking the operation down with it.

None of these requirements show up in a 15-minute demo. All of them show up months after go-live, at 3 a.m., during an unexpected spike. That moment is what infrastructure is built for.

Why "a good model" isn't enough

There's a comfortable illusion in the market: that conversational AI is, deep down, an API call to a language model. Connect to the provider, write a prompt, done.

That view works until the first contact with the reality of a serious operation. The model is the most visible and, paradoxically, the most replaceable part of the system. What truly determines whether 4 million messages reach their destination with quality is the engineering around it:

  • The context layer that decides, for every message, what the model needs to know (history, CRM data, business rules, brand policy) without blowing up cost or the context window.
  • The orchestration that coordinates multiple agents, tools and integrations without becoming a fragile tangle (see our article on AI agent orchestration).
  • The data layer that records every interaction with integrity, queryable and auditable.
  • The routing and failover that keep the operation standing when a provider degrades or a spike hits.
  • The observability that shows, in real time, what's happening, before the customer complains.

Switching models is an afternoon's decision. Building the infrastructure that makes millions of messages flow safely is the work of years. That infrastructure is where a platform's seriousness lives.

The invisible infrastructure behind every conversation

It's worth opening up what sustains this volume. Not to show off engineering, but because it's exactly what a large company needs to see before trusting its operation to the conversation.

Horizontal scale, not heroics

Handling volume can't depend on a bigger machine or an attentive on-call engineer. Our architecture is designed to scale horizontally: when traffic grows, the system adds capacity elastically and removes it when the spike passes. Campaign peaks, billing seasonality and support crises are treated as the normal state of things, because at scale they are.

Latency as a requirement, not luck

Conversation is real time. A delay of a few seconds turns a good answer into a bad experience. That's why latency is, for us, a budget that is measured and defended: every step in the chain (context retrieval, model call, persistence, integration) has a monitored time cost, and regressions are treated as production bugs, not details.

Data isolation

For public-sector, healthcare, financial and industrial companies, where the data lives and who accesses it isn't a preference: it's a legal requirement. We operate with per-customer isolation, encryption and audit trails designed to meet data-protection requirements (LGPD). The AI can read from and write to the customer's systems when that's necessary to resolve an issue, but always within explicit, logged boundaries.

Observability: seeing before it hurts

You can't run 4 million messages in the dark. Every conversation leaves a trace: response-quality metrics, sentiment, first-response time, resolution rate, escalations. When something starts to degrade, we see it on the chart before it becomes a complaint. Operating blind at scale isn't courage: it's negligence.

Graceful degradation and recovery

At scale, failure isn't a hypothesis: it's statistics. A slow provider, an integration that drops, a queue that fills up. The right question isn't "how do we avoid every failure" (impossible), but "how do we fail without taking the operation down". We work with redundancy, resilient queues and fallback paths so a degraded part doesn't contaminate the whole, and so the system returns to normal without manual intervention.

Robustness is a product decision, not a technical detail

It's tempting to treat reliability as a backstage matter, as though it were something the engineering team sorts out while the "real" product is the features. We strongly disagree.

For a company that puts service, sales and billing into the conversation, robustness is the feature. What good is the smartest agent on the market if it goes unavailable at the peak, answers slowly under load, or loses context halfway through a conversation? Intelligence only has value if it arrives, on time, every time.

That's why we treat availability, latency and data integrity as first-class product requirements, with the same seriousness we give a new AI capability. When a large company evaluates a conversational platform, it isn't buying a pretty demo. It's buying the confidence that the operation will still be standing two years from now, at twice the volume, without becoming a headache.

What this means for large enterprises

If you lead operations, technology or service at a high-volume company, the 4-million-message mark says three practical things:

  1. We're past the experiment stage. We're not validating whether conversational AI works: we run it in production, at scale, every day. The risk of "being the first to test it" is lower: the product already runs at real volume.
  2. Scale is the native environment, not the exception. Your operation won't "stress" the platform; it enters an environment designed for volume from day one. Growing generally means using more of the same base, without switching platforms.
  3. The conversation can become a critical channel safely. Availability, data isolation, auditing and observability are part of the foundation, not the roadmap. For corporate needs such as SAML SSO, low latency and a dedicated environment, see the Enterprise plan at /precos. That's what lets you move service, sales and billing onto AI without outsourcing the manager's sleep.

In other words: volume is not a substitute for evidence. Any vendor can promise scale on a slide; what matters is showing how the architecture handles spikes, provider failures and degradation. Always ask for availability and latency metrics (for example, p95 of first response) in your evaluation.

The maturity that comes with volume

There's a quiet gain in operating at this level: compound learning. Every million messages shows us patterns no theory delivers: where the AI gets it right on its own, where it needs a handoff, where context makes the difference, where latency weighs. That learning flows back into the product as better answers, leaner flows and sounder architecture decisions.

It's a self-reinforcing cycle: volume demands robust infrastructure; robust infrastructure sustains more volume; more volume generates more learning; learning makes the product better and attracts more volume. Reaching 4 million quality messages doesn't happen by luck: it happens by building the right foundation, in the right order.

Frequently asked questions

What do the "4 million AI messages a month" mean?

It's the volume of messages processed by Tolky's AI each month across real customer operations: support, sales, billing, service and relationships on the channels active today, WhatsApp, webchat, Microsoft Teams and voice (Instagram and other channels are coming soon). Period, channels and the exact definition of a "processed message" can vary by operation; ask the team. These are production interactions, not internal tests.

Why do you talk so much about infrastructure and not just AI?

Because, at scale, the quality of the experience depends more on the engineering around the model than on the model itself. Availability, latency under load, data isolation, observability and fault recovery are what keep millions of conversations reliable. Without that foundation, a good model delivers a good demo and a fragile operation.

Can the platform handle campaign peaks and seasonality?

It was designed for that. The architecture scales horizontally and treats peaks as the normal state. Capacity is added elastically at the peak and removed afterward, keeping latency stable under load.

What about security and compliance?

We operate with per-customer data isolation, encryption and audit trails designed to meet data-protection law (LGPD). The AI accesses and writes to the customer's systems only within explicit, logged boundaries. The Enterprise plan includes SAML SSO; for certifications and data residency, ask the team (we do not claim SOC 2, ISO 27001 or Brazil data residency).

My operation is large. Will I need to replatform as I grow?

That's not expected. The environment was designed for volume from the start; growing means using more of the same foundation, not switching it.

Next steps

4 million messages a month is a milestone, and for us above all a commitment: to treat relational AI as the critical infrastructure it has become for our customers.

Share

Tags

About the author

Marlos Carmo

Founder of Tolky

Marlos Carmo is an AI entrepreneur and founder of Tolky, the conversational-era infrastructure and AI CRM that unifies intelligent service, multi-channel support (such as WhatsApp and voice), live CRM, and operational intelligence in a single ecosystem. He is a finalist for the SXSW Innovation Awards and a member of Francesco's Economy, a global network of young entrepreneurs focused on innovation and social impact. He works connecting Artificial Intelligence and digital transformation in projects for large organizations.

Next step

Want to bring this to your operation?

Tati, Tolky's AI assistant, understands your scenario in a few questions, suggests the best path and connects you with someone from the team, with full context.

Read also