IBM Granite is a family of open foundation models developed by IBM for business applications, covering language, code, vision, speech, embeddings, time series and safety classification. Among AI models & assistants, the Granite models stand out less for raw benchmark leadership than for their enterprise orientation: permissive licensing, documented training data, small and efficient sizes, and governance features that matter to regulated organizations.
In the wider field of artificial intelligence, Granite occupies the space of compact open-weight models alongside families such as Llama, Mistral, Gemma, Phi and Qwen. Most Granite releases are published under the Apache 2.0 license, can run locally or in the cloud, and are also offered through IBM watsonx with contractual intellectual property protection for customers.
How the Granite family developed
Granite began in 2023 as a set of proprietary models inside IBM watsonx and became an open-weight family in 2024, starting with the Granite Code models. Each subsequent generation broadened the scope from text to code, vision, speech and specialist tasks while keeping parameter counts deliberately modest.
The table below summarizes the main generations and the change each one introduced. Exact model lists vary by release, so the entries describe the defining features rather than every checkpoint.
| Generation | Approximate period | Defining change |
|---|---|---|
| Granite 13B | 2023 | First proprietary language models on watsonx |
| Granite Code | 2024 | Open code models from 3B to 34B parameters |
| Granite 3.0 to 3.3 | Late 2024 to 2025 | Dense and mixture-of-experts models, 128K context, vision, speech and a reasoning toggle |
| Granite 4.0 | Late 2025 | Hybrid Mamba-2 and transformer architecture for lower memory use |
Throughout this history, IBM has emphasized transparency about training data and filtering, publishing technical reports that describe data sources, deduplication and governance reviews. This openness is a deliberate contrast with models whose training corpora remain undisclosed.
Architecture and the hybrid design of Granite 4.0
Granite 4.0 combines Mamba-2 state space layers with a smaller share of transformer attention layers. State space layers process a sequence with memory that grows linearly with length, whereas attention compares every token with every other token and needs a key-value cache that grows with context. Mixing the two keeps most of the quality of attention while sharply reducing memory use on long inputs.
Some Granite 4.0 variants also use mixture-of-experts routing, in which only a fraction of the parameters is active for each token. The result is a model that stores more knowledge than it computes per step. The table below compares the main design elements.
| Design element | What it does | Practical benefit |
|---|---|---|
| Dense transformer | All parameters active for every token | Predictable behavior and broad tool support |
| Mixture of experts | Routes each token to a subset of experts | Lower compute per token at a given capacity |
| Mamba-2 layers | Linear-time sequence processing | Much smaller memory footprint on long context |
| Hybrid stack | Mostly Mamba-2 with periodic attention | Long-context efficiency with retained precision |
IBM reports that the hybrid design can cut inference memory by more than half compared with conventional transformers of similar size, especially with long documents and many concurrent sessions. The actual saving depends on context length, batch size and the serving engine.
Model types within the family
Granite is not a single model but a portfolio of task-specific models that share naming and licensing. Organizations typically combine several of them in one system, for example a language model for generation, an embedding model for retrieval and a guardian model for safety checks.
| Branch | Purpose | Typical use |
|---|---|---|
| Granite language models | General instruction following and tool calling | Assistants, summarization, agents |
| Granite Code | Code generation and explanation | Developer tools, legacy code modernization |
| Granite Guardian | Detection of harmful content and hallucination | Input and output filtering in RAG systems |
| Granite Embedding | Dense vector representations | Semantic search and retrieval |
| Granite Vision | Image and document understanding | Tables, charts and scanned forms |
| Granite Speech | Speech recognition and translation | Transcription pipelines |
| Granite Time Series | Compact forecasting models | Demand, sensor and capacity forecasts |
The Granite Guardian models deserve particular attention, because they act as classifiers for risk rather than as chat models. They flag jailbreak attempts, harmful content, and answers that are not supported by retrieved context, which makes them useful even next to models from other vendors.
How Granite compares with other open model families
Granite competes mainly in the small and mid-size open-weight segment. Its positioning differs from families optimized primarily for benchmark rankings or consumer chat.
| Family | Main strength | Licensing emphasis |
|---|---|---|
| IBM Granite | Enterprise governance and efficiency | Apache 2.0 with optional indemnity on watsonx |
| Llama | Large ecosystem and community tooling | Custom community license with conditions |
| Mistral | Strong small models and European presence | Mix of Apache 2.0 and commercial licenses |
| Qwen | Broad multilingual and coding strength | Mostly Apache 2.0 for open releases |
| Phi and Gemma | Very compact models for local use | Permissive or custom terms by release |
On public leaderboards, Granite models generally perform competitively within their size class rather than leading the frontier. Their advantage appears in total cost of operation and compliance, where license clarity, documented data and certified management processes reduce legal and procurement friction.
Governance, licensing and limitations
IBM positions Granite as a governance-first model family. The company states that Granite 4.0 was covered by an ISO/IEC 42001 certified AI management system, the international standard for managing artificial intelligence responsibly, and model checkpoints are cryptographically signed so that users can verify their origin.
- Capability ceiling: the smaller Granite models trail frontier systems on complex reasoning, advanced mathematics and open-ended creative writing.
- Language coverage: support is strongest in English and a set of major languages, and quality drops for low-resource languages.
- Indemnity scope: intellectual property protection applies to use through IBM offerings under contract, not to every self-hosted copy.
- Tooling maturity: the hybrid architecture requires serving engines that support Mamba-2 layers, which older inference stacks may lack.
A frequent mistake is to treat governance documentation as a guarantee of correct output. Granite models can still hallucinate, so production systems should combine them with retrieval, evaluation and output checks.
Stripped to the essentials
IBM Granite is a set of AI models that IBM gives away for free use, including in commercial products. Some models write text, some write code, some read documents or listen to speech, and some check whether an answer is safe.
They are not the biggest or flashiest models available. Their appeal is that they are small enough to run cheaply, their rules of use are simple, and IBM explains how they were trained. For a company that must justify every tool to auditors, that clarity can matter more than a few extra points on a leaderboard.
Where it is applied
A regional insurer wants an internal assistant that answers claims handlers using policy documents, but its compliance team will not send customer data to an external service. The IT department deploys a small Granite language model on its own servers, pairs it with Granite Embedding for document search and places Granite Guardian in front of every answer to flag responses that are not supported by the retrieved policy text.
Because the models carry an Apache 2.0 license and published training documentation, the procurement review takes weeks rather than months. In the pilot, handlers use the assistant to locate clauses and draft summaries, while every flagged answer is routed to a senior reviewer, which keeps the error rate within the limits agreed with the compliance team.