A large language model (LLM) is a neural network trained on very large text corpora to predict the next token in a sequence, and through that single objective it learns to generate, summarize, translate, classify and reason over language. Almost every modern LLM is built on the transformer architecture, and its scale is measured in billions of learned parameters.
In artificial intelligence, the LLM is the core component of chat assistants, coding tools, retrieval systems and AI agents. Its practical behavior depends on four things: the training data, the model size, the context window it can read at once and the post-training that turns a raw text predictor into a useful assistant.
How a large language model generates text
An LLM generates text one token at a time in a loop called autoregressive decoding. Each new token is appended to the input, and the model runs again to predict the next one until it emits a stop token or reaches a length limit.
Inside each step, the input passes through a fixed pipeline. Tokenization splits text into subword units, an embedding layer maps each token to a vector, and dozens of stacked transformer layers refine those vectors with self-attention, which lets every token weigh every earlier token in the context. The final layer produces a probability distribution over the whole vocabulary, and a sampling rule picks the next token.
The table below follows one generation step from raw text to output.
| Stage | What happens | Key setting |
|---|---|---|
| Tokenization | Text is split into subword tokens from a fixed vocabulary | Vocabulary size, often 32,000 to 256,000 tokens |
| Embedding | Each token ID becomes a dense vector | Hidden dimension, often 2,048 to 16,384 |
| Transformer layers | Self-attention and feed-forward blocks refine every vector | Number of layers, roughly 20 to 120 |
| Output projection | The last vector is scored against every vocabulary entry | Logits converted to probabilities by softmax |
| Sampling | One token is chosen from the distribution | Temperature, top-p, top-k |
Because the model only ever predicts a plausible continuation, it has no separate store of verified facts. Everything it knows is encoded statistically in its weights, which explains both its fluency and its tendency to produce confident errors.
How LLMs are trained
An LLM is trained in two broad phases: large-scale pretraining, which builds general language competence, and post-training, which shapes the model into a reliable assistant. Pretraining consumes almost all of the compute, while post-training determines most of the behavior users notice.
Pretraining is self-supervised: the next token in existing text serves as the label, so no human annotation is needed. Current open models are typically pretrained on several trillion to more than fifteen trillion tokens of web text, books, code and curated data. Post-training then adds instruction examples and human or AI preference signals.
| Stage | Training data | Objective | Result |
|---|---|---|---|
| Pretraining | Trillions of tokens of unlabeled text and code | Predict the next token | Base model with broad knowledge |
| Supervised fine-tuning | Thousands to millions of prompt and response pairs | Imitate high-quality answers | Model that follows instructions |
| Preference alignment | Ranked or paired responses | Prefer helpful, harmless answers via RLHF or DPO | Assistant with safer, more useful style |
| Reasoning training | Problems with checkable answers | Reward correct multi-step solutions | Better math, coding and planning |
A practical consequence is the knowledge cutoff. The model knows nothing about events after its training data was collected unless that information is supplied in the prompt, for example through retrieval.
Key parameters that define an LLM
A handful of measurable properties determine what an LLM can do, what it costs to run and what hardware it needs. The table lists the most important ones with typical ranges for current models.
| Property | Typical range | What it affects |
|---|---|---|
| Parameter count | About 1 billion to over 1 trillion | Capability, memory use, speed |
| Training tokens | Roughly 1 to 15+ trillion | Breadth of knowledge and skill |
| Context window | 4,000 to over 1,000,000 tokens | How much text fits in one request |
| Weight precision | 16-bit, 8-bit or 4-bit | Memory footprint and slight quality loss |
| Architecture | Dense or mixture of experts | Compute per token versus total size |
Memory needs follow directly from size and precision. A 7-billion-parameter model at 16-bit precision needs 2 bytes per parameter, or about 14 GB for the weights alone; quantized to 4 bits, the same model needs about 3.5 GB. A 70-billion-parameter model at 16-bit precision needs about 140 GB, which is why such models run on multiple GPUs or in quantized form. The key-value cache for long contexts adds further memory on top of the weights.
In a mixture-of-experts model, a router activates only a few expert subnetworks for each token. The total parameter count can therefore be very large while the active parameters per token, and thus the compute cost, stay much smaller.
LLMs compared with related model types
Not every transformer language model is an LLM in the everyday sense. The term usually refers to large decoder-only generative models, while smaller or encoder-based models serve narrower tasks.
| Model type | Structure | Main strength | Typical use |
|---|---|---|---|
| Decoder-only LLM | Predicts tokens left to right | Open-ended generation | Chat, writing, code, agents |
| Encoder-only model | Reads the whole input in both directions | Text understanding | Classification, embeddings, search |
| Encoder-decoder model | Encodes input, then decodes output | Input-to-output mapping | Translation, summarization |
| Small language model | Usually under about 10 billion parameters | Low cost and on-device use | Edge apps, narrow tasks |
| Multimodal model | LLM with image or audio encoders | Mixed inputs | Document and image analysis |
The boundary between these categories is fluid. Many current LLMs are natively multimodal, and a well-trained small model can outperform a much larger older model on specific benchmarks.
Limitations and failure modes of LLMs
The main limitations of an LLM follow from its design as a statistical next-token predictor with a finite context. Experienced teams plan around them rather than hoping a larger model will remove them.
| Failure | Cause | Mitigation |
|---|---|---|
| Hallucination | Fluent guessing when knowledge is missing | Retrieval, citations to supplied text, verification |
| Outdated answers | Fixed knowledge cutoff | Supply current data in the prompt |
| Missed details in long inputs | Uneven attention across very long contexts | Chunking, focused prompts, reranking |
| Prompt injection | Instructions hidden in untrusted input | Input isolation, least-privilege tools |
| Inconsistent output | Sampling randomness and prompt sensitivity | Low temperature, structured output, tests |
- An LLM does not look facts up unless a retrieval or search step is added; it reconstructs them from patterns.
- Benchmark scores do not guarantee task performance; evaluation on the actual workload is essential.
- Cost scales with tokens, so long prompts and verbose outputs raise both latency and spending.
The short version
A large language model is a program that has read an enormous amount of text and learned which words tend to follow which. When you ask it something, it writes the answer one small piece at a time, each time choosing the piece that best fits everything written so far.
This simple skill, practiced at huge scale, lets it write, summarize, translate and explain. It is a bit like a very well-read colleague who answers from memory: usually helpful, sometimes confidently wrong, and only as current as the last thing it read. Check important answers before relying on them.
Where it makes a difference
A mid-sized insurance company receives about 1,200 customer emails per day about claims, policy changes and billing. Agents previously wrote every reply from scratch, which took around nine minutes per email and produced uneven wording across the team.
The company connects an LLM to its policy documents through a retrieval step and asks it to draft a reply for each email, citing the relevant policy clause. A human agent reviews and edits every draft before sending. Temperature is set low so that similar questions receive consistent answers, and emails that mention legal action are routed straight to senior staff without a draft.
After a three-month pilot, average handling time drops to about six minutes per email, a saving of roughly one third, and quality reviewers find fewer contradictions between replies. The decisive design choice is that the model drafts while people decide, which keeps hallucinated policy details from ever reaching a customer.