A code LLM is a large language model specialized for source code: it is trained or fine-tuned on programs, documentation and developer conversations so that it can generate, complete, explain, translate and repair code. Among language models in artificial intelligence, a code LLM (often simply called a code model) is distinguished less by its architecture than by its training data, tokenizer choices and evaluation targets, which are tuned to the strict syntax and executable semantics of programming languages.
Code LLMs power inline completion in editors, chat assistants that answer programming questions, automated review tools and coding agents that edit repositories and run tests. Their output can be checked by a compiler or a test suite, which makes code one of the few domains where model quality can be measured objectively at scale.
How a code LLM learns to write programs
A code LLM learns programs the same way a general LLM learns prose: through next-token prediction on a very large corpus, usually with a decoder-only transformer. The difference lies in the corpus, which is dominated by permissively licensed public repositories, issue threads, commit messages, technical documentation and question-and-answer pages, typically filtered for duplicates, generated files and low-quality code.
Several training objectives are specific to code. Fill-in-the-middle (FIM) training rearranges a file into prefix, suffix and middle segments so the model learns to insert code at the cursor while seeing the text after it. Repository-level training concatenates related files in dependency order, teaching the model how a function defined in one file is called in another. Later stages add instruction tuning on paired requests and solutions and, increasingly, reinforcement learning from execution feedback, where the reward depends on whether generated code compiles and passes unit tests.
The following table summarizes the main training stages and what each contributes to coding ability.
| Stage | Data used | Capability gained |
|---|---|---|
| Pretraining | Mixed code and natural language, often trillions of tokens | Syntax, idioms, library knowledge |
| Fill-in-the-middle | Files split into prefix, middle and suffix | Cursor-aware completion in editors |
| Long-context training | Whole repositories or concatenated files | Cross-file references and project structure |
| Instruction tuning | Request and solution pairs, chat transcripts | Following natural-language coding requests |
| Execution-based reinforcement learning | Problems with unit tests or verifiers | Higher functional correctness, self-correction |
Tokenization and context in code models
Tokenization matters more for code than for prose because whitespace and symbols carry meaning. Code-oriented tokenizers, usually variants of byte-pair encoding, add tokens for runs of spaces and tabs, common operators and frequent identifiers, which reduces the number of tokens per file and keeps indentation in languages such as Python intact.
The context window determines how much of a project the model can see at once. Completion tools assemble a prompt from the current file, open tabs and retrieved snippets, while agents read files on demand. Even with windows of 100,000 tokens or more, a large monorepo does not fit, so practical systems combine the model with retrieval, code search and symbol indexes. Models also show the lost-in-the-middle effect, attending less reliably to material buried in the center of a long prompt.
Tasks a code LLM performs
Code LLMs serve a range of tasks that differ in latency budget, context size and how errors are caught. The table below compares the most common ones.
| Task | Typical latency need | How errors are caught |
|---|---|---|
| Inline completion | Under about 300 ms | Developer reads and accepts or rejects |
| Function generation | A few seconds | Unit tests, type checker, review |
| Code explanation | A few seconds | Developer judgment only |
| Language translation | Seconds to minutes | Behavioral tests on both versions |
| Bug fixing and refactoring | Seconds to minutes | Regression test suite, static analysis |
| Agentic repository tasks | Minutes to hours | Continuous integration, human pull request review |
Latency drives model choice. Inline completion favors small, fast models served close to the editor, often with a few billion parameters, while multi-step agentic work favors larger reasoning models that spend more test-time compute planning, running tools and revising their own output.
How code LLMs compare with general LLMs
A code LLM differs from a general-purpose LLM mainly in emphasis rather than kind. Modern frontier general models are trained on large amounts of code and often match or exceed dedicated code models, while specialized code models remain attractive when cost, latency, privacy or self-hosting matter.
| Aspect | Specialized code LLM | General-purpose LLM |
|---|---|---|
| Training mix | Mostly code and technical text | Broad text with substantial code |
| Completion support | Native fill-in-the-middle | Often chat-only interface |
| Model size | Frequently small to mid-sized | Often very large |
| Deployment | Common as open-weight models on local hardware | Mostly hosted APIs |
| Non-coding knowledge | Limited | Broad domain and world knowledge |
The practical consequence is that teams often run two tiers: a small local model for keystroke-level completion and a large hosted model for design questions, complex refactoring and agentic work.
Measuring code LLM quality
Code LLMs are evaluated chiefly by functional correctness: generated code is executed against hidden tests. The standard metric is pass@k, the probability that at least one of k sampled solutions passes all tests; pass@1 reflects single-attempt reliability, while pass@10 reflects how useful the model is when several candidates can be filtered.
Benchmarks progressed from short function-level problems, such as HumanEval and MBPP, to repository-level benchmarks such as SWE-bench, in which the model must resolve real issues in open-source projects. Older benchmarks are now largely saturated and contaminated, because their solutions have leaked into training corpora, so results on them overstate real-world ability.
| Approach | What it measures | Main weakness |
|---|---|---|
| Function-level benchmarks | Correctness on short, self-contained problems | Saturation and data contamination |
| Repository-level benchmarks | Resolving real issues across many files | Expensive to run, flaky test environments |
| Acceptance rate in editors | Share of suggestions developers keep | Rewards plausible rather than correct code |
| Internal task suites | Performance on the team’s own codebase | Small samples, costly to maintain |
Limitations and risks of code LLMs
The central risk is that a code LLM produces plausible code that is subtly wrong. Because output is fluent and idiomatic, defects in edge cases, concurrency and error handling are easy to miss during review.
- Hallucinated APIs and packages: the model invents functions or dependency names; attackers can register invented package names, a risk known as package hallucination squatting.
- Insecure patterns: training data contains vulnerable code, so models can reproduce SQL injection, weak cryptography or missing input validation.
- Outdated knowledge: the knowledge cutoff means newer library versions and deprecations may be unknown to the model.
- Licensing and provenance: generated snippets can resemble licensed code, which concerns organizations with strict compliance rules.
- Skill and review erosion: heavy reliance can reduce careful reading of generated changes, so mandatory review and automated tests remain essential.
A non-technical view
A code LLM is a writing assistant for software. It has read an enormous amount of existing code, so when a programmer starts typing or describes what is needed, it suggests the next lines or a whole working piece. It predicts likely code, not guaranteed correct code.
Think of it as a fast, well-read junior colleague. It saves a great deal of typing and can explain unfamiliar code, but its work must still be checked, ideally by running tests, before anyone relies on it.
A concrete example
A small backend team maintains a payment reconciliation service of about 80,000 lines of Python. They run a small open-weight code model on a local server for inline completion, so that proprietary code never leaves the company network, and use a larger hosted model for refactoring tasks with strict data-handling terms.
When the team migrates from one database library version to the next, an engineer asks the larger model to update 40 query functions. The model rewrites them in minutes, but the test suite fails on three functions where the model used a parameter that no longer exists in the new version. The engineer feeds the error output back, the model corrects the calls, and all tests pass.
The outcome is a migration finished in one day instead of roughly a week, with every change reviewed in a pull request. The team keeps the rule that no generated code merges without passing tests.