Chatbot

Also known as: chat bot · conversational AI

On this page 7

A chatbot is a software system that conducts a dialogue with people in natural language, through text or voice, to answer questions, complete tasks or keep a conversation going. In artificial intelligence, and specifically in speech & language AI, the term covers everything from scripted decision trees to conversational AI built on large language models, so the word describes an interface role rather than a single technology.

Every chatbot combines three capabilities: it must understand the user turn, decide what to do next and produce a reply. How each capability is implemented determines what the chatbot can handle, how predictable it is and how much it costs to build and operate.

How a chatbot processes a conversation

A chatbot processes each user message through a pipeline of understanding, state tracking, decision and generation. In classic designs these stages are separate modules; in LLM-based chatbots a single model performs most of them implicitly, guided by instructions and retrieved context.

The following table maps the classic pipeline stages to their purpose and to the way a modern language model handles the same job.

StagePurposeClassic implementationLLM-based implementation
Natural language understandingInterpret the user messageIntent recognition and slot fillingImplicit in the prompt and model
Dialogue state trackingRemember what is known so farExplicit slots and variablesConversation history in the context window
Dialogue managementChoose the next actionRules, flows or a trained policyModel reasoning plus tool calls
Backend integrationFetch data or perform actionsHard-coded API callsFunction calling with validated arguments
Natural language generationPhrase the replyTemplates with filled variablesFree-form generated text
Stages of a chatbot pipeline

Voice chatbots add speech-to-text before understanding and text-to-speech after generation. They also need turn detection and barge-in handling, because a spoken conversation has no send button that marks the end of a turn.

Types of chatbots and their architectures

Chatbots fall into four broad architectural families, which differ in flexibility, control and failure behavior. Many production systems are hybrids that route predictable requests to fixed flows and open-ended requests to a language model.

ArchitectureHow it respondsStrengthMain weakness
Rule-basedPattern matching and decision treesFully predictable and auditableBreaks on unexpected phrasing
Intent-basedClassifier maps input to predefined intentsRobust to paraphrase within scopeNeeds labeled examples per intent
Retrieval-basedSelects the best stored answerAnswers stay approved and consistentCannot compose new answers
Generative (LLM)Generates replies token by tokenHandles open questions fluentlyCan hallucinate or drift off policy
Main chatbot architectures compared

A widely used pattern is retrieval-augmented generation, where the chatbot first searches a document collection and then asks the model to answer only from the retrieved passages. This grounds replies in current, organization-specific content and reduces, but does not eliminate, fabricated answers.

A chatbot is the conversational front end, while neighboring concepts describe either components inside it or broader systems around it. Confusing them leads to wrong expectations about what a project will deliver.

ConceptRelation to a chatbotKey distinction
Question answeringA capability a chatbot may useAnswers single questions without dialogue state
Virtual assistantA chatbot with broad personal tasksIntegrates calendars, devices and accounts
AI agentMay sit behind a chatbot interfacePlans multi-step actions with tools autonomously
Interactive voice responseA telephone predecessorMenu-driven keypad or limited speech input
Large language modelA possible engine inside a chatbotA model, not a product with goals and guardrails
Chatbot and neighboring concepts

The distinction between a chatbot and an agent is becoming blurred. A useful working rule is that a chatbot responds within a conversation, whereas an agent pursues a goal across several actions, possibly without a human reply between steps.

Designing and evaluating a chatbot

Good chatbot design starts with a clearly bounded scope: the tasks it must complete, the questions it must refuse and the point at which it hands over to a human. Without explicit boundaries, users test the system with requests it was never meant to handle, and satisfaction drops quickly.

Evaluation combines automatic metrics with human review. The table below lists common measures and what each one reveals.

MetricWhat it measuresTypical use
Containment rateShare of sessions resolved without a humanCustomer service cost planning
Task completion rateShare of goals fully achievedTransactional bots such as bookings
Intent accuracyCorrect classification of user goalsIntent-based systems
GroundednessShare of claims supported by sourcesRetrieval-augmented chatbots
Escalation qualityWhether handover carries full contextHybrid human and bot support
Common chatbot evaluation metrics

A high containment rate is not automatically good. If users abandon the chat in frustration, the session may count as contained even though the problem was never solved, so containment must be read together with resolution and satisfaction.

Limitations, risks and common mistakes

The main risks of a chatbot come from misunderstanding, fabrication and misuse. Generative chatbots add a specific failure mode: they produce fluent but false statements with the same confident tone as correct ones.

  • Hallucination: the model invents policies, prices or citations; grounding, answer constraints and refusal rules reduce it.
  • Prompt injection: user or document text overrides the instructions; the chatbot must treat retrieved content as data and restrict tool permissions.
  • Data leakage: personal or confidential data appears in replies or logs; minimization and access control are required.
  • Dead-end flows: users cannot reach a person; a visible escape route to a human agent prevents abandonment.
  • Undisclosed automation: users believe they talk to a person; regulations such as the EU AI Act require that people are told when they interact with an AI system.

A frequent project mistake is to measure the demo instead of the long tail. A chatbot that handles the twenty most common questions perfectly may still fail on the thousands of rare phrasings that real users produce, so testing on real transcripts matters more than scripted demonstrations.

In everyday language

A chatbot is a computer program you can talk to by typing or speaking, much like sending messages to a helpful clerk. It tries to work out what you want, looks up the right information or performs the action, and then writes back in ordinary sentences.

Older chatbots follow a fixed script, a bit like a phone menu dressed up as a conversation. Newer ones use language models and can answer almost any wording, but they can also sound sure while being wrong. The best chatbots know their limits and pass you to a person when the question goes beyond them.

Seen in the real world

A regional electricity supplier receives about 40,000 customer messages a month, most of them about bills, meter readings and outage status. It deploys a hybrid chatbot on its website and mobile app: fixed flows handle meter submissions and payment plans, while a language model with retrieval over the tariff documents answers open questions about prices and contract terms.

The chatbot verifies the customer through the logged-in session, reads outage data from the network system and hands complex complaints to a human agent together with a summary of the conversation. After three months, the supplier reports that roughly half of the conversations are resolved without an agent, average waiting time for the remaining cases falls, and weekly reviews of sampled transcripts catch and correct the few answers that quoted an outdated tariff.