A chatbot is a software system that conducts a dialogue with people in natural language, through text or voice, to answer questions, complete tasks or keep a conversation going. In artificial intelligence, and specifically in speech & language AI, the term covers everything from scripted decision trees to conversational AI built on large language models, so the word describes an interface role rather than a single technology.
Every chatbot combines three capabilities: it must understand the user turn, decide what to do next and produce a reply. How each capability is implemented determines what the chatbot can handle, how predictable it is and how much it costs to build and operate.
How a chatbot processes a conversation
A chatbot processes each user message through a pipeline of understanding, state tracking, decision and generation. In classic designs these stages are separate modules; in LLM-based chatbots a single model performs most of them implicitly, guided by instructions and retrieved context.
The following table maps the classic pipeline stages to their purpose and to the way a modern language model handles the same job.
| Stage | Purpose | Classic implementation | LLM-based implementation |
|---|---|---|---|
| Natural language understanding | Interpret the user message | Intent recognition and slot filling | Implicit in the prompt and model |
| Dialogue state tracking | Remember what is known so far | Explicit slots and variables | Conversation history in the context window |
| Dialogue management | Choose the next action | Rules, flows or a trained policy | Model reasoning plus tool calls |
| Backend integration | Fetch data or perform actions | Hard-coded API calls | Function calling with validated arguments |
| Natural language generation | Phrase the reply | Templates with filled variables | Free-form generated text |
Voice chatbots add speech-to-text before understanding and text-to-speech after generation. They also need turn detection and barge-in handling, because a spoken conversation has no send button that marks the end of a turn.
Types of chatbots and their architectures
Chatbots fall into four broad architectural families, which differ in flexibility, control and failure behavior. Many production systems are hybrids that route predictable requests to fixed flows and open-ended requests to a language model.
| Architecture | How it responds | Strength | Main weakness |
|---|---|---|---|
| Rule-based | Pattern matching and decision trees | Fully predictable and auditable | Breaks on unexpected phrasing |
| Intent-based | Classifier maps input to predefined intents | Robust to paraphrase within scope | Needs labeled examples per intent |
| Retrieval-based | Selects the best stored answer | Answers stay approved and consistent | Cannot compose new answers |
| Generative (LLM) | Generates replies token by token | Handles open questions fluently | Can hallucinate or drift off policy |
A widely used pattern is retrieval-augmented generation, where the chatbot first searches a document collection and then asks the model to answer only from the retrieved passages. This grounds replies in current, organization-specific content and reduces, but does not eliminate, fabricated answers.
How a chatbot differs from related systems
A chatbot is the conversational front end, while neighboring concepts describe either components inside it or broader systems around it. Confusing them leads to wrong expectations about what a project will deliver.
| Concept | Relation to a chatbot | Key distinction |
|---|---|---|
| Question answering | A capability a chatbot may use | Answers single questions without dialogue state |
| Virtual assistant | A chatbot with broad personal tasks | Integrates calendars, devices and accounts |
| AI agent | May sit behind a chatbot interface | Plans multi-step actions with tools autonomously |
| Interactive voice response | A telephone predecessor | Menu-driven keypad or limited speech input |
| Large language model | A possible engine inside a chatbot | A model, not a product with goals and guardrails |
The distinction between a chatbot and an agent is becoming blurred. A useful working rule is that a chatbot responds within a conversation, whereas an agent pursues a goal across several actions, possibly without a human reply between steps.
Designing and evaluating a chatbot
Good chatbot design starts with a clearly bounded scope: the tasks it must complete, the questions it must refuse and the point at which it hands over to a human. Without explicit boundaries, users test the system with requests it was never meant to handle, and satisfaction drops quickly.
Evaluation combines automatic metrics with human review. The table below lists common measures and what each one reveals.
| Metric | What it measures | Typical use |
|---|---|---|
| Containment rate | Share of sessions resolved without a human | Customer service cost planning |
| Task completion rate | Share of goals fully achieved | Transactional bots such as bookings |
| Intent accuracy | Correct classification of user goals | Intent-based systems |
| Groundedness | Share of claims supported by sources | Retrieval-augmented chatbots |
| Escalation quality | Whether handover carries full context | Hybrid human and bot support |
A high containment rate is not automatically good. If users abandon the chat in frustration, the session may count as contained even though the problem was never solved, so containment must be read together with resolution and satisfaction.
Limitations, risks and common mistakes
The main risks of a chatbot come from misunderstanding, fabrication and misuse. Generative chatbots add a specific failure mode: they produce fluent but false statements with the same confident tone as correct ones.
- Hallucination: the model invents policies, prices or citations; grounding, answer constraints and refusal rules reduce it.
- Prompt injection: user or document text overrides the instructions; the chatbot must treat retrieved content as data and restrict tool permissions.
- Data leakage: personal or confidential data appears in replies or logs; minimization and access control are required.
- Dead-end flows: users cannot reach a person; a visible escape route to a human agent prevents abandonment.
- Undisclosed automation: users believe they talk to a person; regulations such as the EU AI Act require that people are told when they interact with an AI system.
A frequent project mistake is to measure the demo instead of the long tail. A chatbot that handles the twenty most common questions perfectly may still fail on the thousands of rare phrasings that real users produce, so testing on real transcripts matters more than scripted demonstrations.
In everyday language
A chatbot is a computer program you can talk to by typing or speaking, much like sending messages to a helpful clerk. It tries to work out what you want, looks up the right information or performs the action, and then writes back in ordinary sentences.
Older chatbots follow a fixed script, a bit like a phone menu dressed up as a conversation. Newer ones use language models and can answer almost any wording, but they can also sound sure while being wrong. The best chatbots know their limits and pass you to a person when the question goes beyond them.
Seen in the real world
A regional electricity supplier receives about 40,000 customer messages a month, most of them about bills, meter readings and outage status. It deploys a hybrid chatbot on its website and mobile app: fixed flows handle meter submissions and payment plans, while a language model with retrieval over the tariff documents answers open questions about prices and contract terms.
The chatbot verifies the customer through the logged-in session, reads outage data from the network system and hands complex complaints to a human agent together with a summary of the conversation. After three months, the supplier reports that roughly half of the conversations are resolved without an agent, average waiting time for the remaining cases falls, and weekly reviews of sampled transcripts catch and correct the few answers that quoted an outdated tariff.