Skip to content

Conversational AI Integration with Legacy Systems: CRM, ERP, APIs

Discover the engineering behind Autonomous AI Agents: how Language Models (LLMs) communicate in real time with legacy CRMs and ERPs through corporate APIs.

Marlos Carmo

Founder of Tolky

Updated on November 15, 2026

5 min read
Conversational AI Integration with Legacy Systems: CRM, ERP, APIs

Share

When the impact of AI on the enterprise market is discussed, the focus usually lands on language fluency. For CTOs and software architects, the real conversation is different: an AI that only "talks" well is of little use at scale. If a customer wants to cancel a plan, the AI has to query the billing system, check contract rules in the ERP and record the cancellation.

This guide shows, behind the scenes, how conversational AI integrates with legacy systems (CRM, ERP, databases). For the business view, read AI without integration becomes an FAQ; for the CRM-specific case, see AI integration with CRM and the AI CRM guide.


How AI learns to act: Function Calling

Historically, connecting a chat to a database required rigid paths: the "My Orders" button fired a fixed GET to /api/orders/{userId}. With state-of-the-art LLMs, the architecture uses Function Calling (tool calling). Besides the conversation history, the agent receives the system's API schema and chooses which tool to call.

The technical sequence:

  1. Intake: the customer writes on WhatsApp: "This month's motorbike installment hasn't cleared yet, can you check?"
  2. Reasoning: the LLM understands the intent: fetch the financial status of the tax ID tied to the phone number for the current month.
  3. Tool selection: among the connected tools, it picks buscar_status_fatura_erp.
  4. Payload: it generates validated JSON (e.g. {"cpf": "12345678900", "mes_referencia": "06-2026"}) and returns it to the orchestration layer.
  5. Execution: orchestration fires the HTTPS request to the ERP and receives the raw response (e.g. "status": "pending_clearing_bank").
  6. Natural reply: the agent translates: "I checked: your payment is processing, waiting for bank clearing. Since the bank slip was paid yesterday, it can take up to 48 business hours."

A tool contract: buscar_status_fatura_erp

ItemDefinition
InputValidated schema: valid tax ID and reference month; any other field is rejected
Per-operation authorizationRead allowed; write (e.g. duplicate invoice, refund) requires its own permission
TimeoutFor example, 8 seconds, with a wait message to the customer while the query runs
ReprocessingIdempotent: an idempotency key prevents the same write from running twice
ConfirmationAfter a write, read the record back to confirm the data was stored
Failure or ERP downThe agent reports the instability, logs a ticket number and escalates to a human with a summary, without inventing an answer

Legacy system challenges

In practice, not every API is modern, clean and fast. Banks, manufacturers and health insurers run on systems 10 or 20 years old. Two obstacles show up often.

SOAP and XML

While the agent works with JSON, many enterprise systems still expose SOAP and XML. The architectural pattern is a middleware layer: the agent always generates and reads JSON; the middleware converts to XML, authenticates against the legacy server and translates the response back. At Tolky, integrations are done through API and MCP agent, depending on the project; ask the team about your system.

Latency

An ERP that takes 12 seconds to answer leaves the customer hanging. The recommended pattern is asynchronous conversation: while the query runs, the agent sends a wait message ("one moment, I'm checking your order") and, if the timeout is exceeded, escalates to a human.


Security, governance and guardrails

The classic CTO question is: "what if the AI starts refunding every customer?". In a well-designed enterprise architecture, the LLM never accesses the database directly: it only requests actions from a layer that validates each one.

  • Restricted authentication: the agent has no admin password. Orchestration applies a token scoped to that customer and session; if the model asks for another customer's data, the API rejects it with a 403 error.
  • Limits in code: a limit such as if (discount > 30) throw Error lives in the integration layer and cannot be influenced by prompts.
  • Human approval: for high-risk actions (cancelling an insurance policy, for example), the agent builds the request and a supervisor approves before execution.

These are architecture patterns. The controls available in each Tolky plan and project (access profiles, audit trail, SSO on Enterprise) are described in plans; the rest should be validated per project.


The intelligence loop: writing back to the CRM

A good integration is not one-way. After a 40-minute service interaction resolving a return, the agent can generate a three-line summary ("customer unhappy with logistics delay, kept the subscription after a 15% discount") and update the customer card in the CRM (Salesforce, HubSpot or Tolky's own CRM) via API. Whoever reads the record later does not need to scroll through hundreds of messages.


Technical verdict

The difficulty of scaling AI in enterprise environments is not model choice, but reliable integration and API orchestration across fragmented architectures. A good LLM does not make up for fragile data pipes or the lack of limits at the API level.

The shift of 2026 is that software no longer just talks: it executes, with rules, permissions and an audit trail.

Want to validate with your ERP or CRM? Bring one system and validate, in a pilot, one read case and one write case. Talk to our team, see the plans or explore the solutions.

Share

Tags

About the author

Marlos Carmo

Founder of Tolky

Marlos Carmo is an AI entrepreneur and founder of Tolky, the conversational-era infrastructure and AI CRM that unifies intelligent service, multi-channel support (such as WhatsApp and voice), live CRM, and operational intelligence in a single ecosystem. He is a finalist for the SXSW Innovation Awards and a member of Francesco's Economy, a global network of young entrepreneurs focused on innovation and social impact. He works connecting Artificial Intelligence and digital transformation in projects for large organizations.

Next step

Want to bring this to your operation?

Tati, Tolky's AI assistant, understands your scenario in a few questions, suggests the best path and connects you with someone from the team, with full context.

Read also