IBM announced the Granite 3.0 model family on October 21, 2024, at its annual TechXchange event, <cite index="1-1">marking the release of the company's most advanced family of artificial intelligence (AI) models to date.</cite>
Model Lineup
<cite index="2-4">The Granite 3.0 release comprises dense, general-purpose large language models (LLMs): Granite-3.0-8B-Instruct, Granite-3.0-8B-Base, Granite-3.0-2B-Instruct, and Granite-3.0-2B-Base.</cite> <cite index="14-11">The generation also introduces Granite's first mixture-of-experts (MoE) models, Granite-3B-A800M-Instruct and Granite-1B-A400-Instruct.</cite> A dedicated safety sub-family, Granite Guardian 3.0, rounds out the release.
<cite index="9-1">Where previous versions of Granite had separate models for English, multilingual, and code use cases, the Granite 3.0 model family consolidates all of these capabilities into one model, while expanding multilingual support to 12 different languages.</cite>
Training and Architecture
<cite index="1-6">The Granite 3.0 models were trained on over 12 trillion tokens drawn from 12 different natural languages and 116 different programming languages, using a novel two-stage training method that leveraged results from several thousand experiments designed to optimize data quality, data selection, and training parameters.</cite> <cite index="4-10">The models were trained using NVIDIA's H100 graphics processing units (GPUs), according to IBM's director of research, Dario Gil.</cite>
Benchmark Performance
<cite index="8-5">On standard academic benchmarks defined by Hugging Face's OpenLLM Leaderboard, the Granite 3.0 8B Instruct model's overall performance leads on average against similar-sized open-source models from Meta and Mistral.</cite> <cite index="8-6,8-7">On IBM's AttaQ safety benchmark, the 8B Instruct model leads across all measured safety dimensions compared to models from Meta and Mistral, and also shows leading performance on core enterprise tasks including retrieval-augmented generation (RAG), tool use, and cybersecurity.</cite>
<cite index="11-1">A new Granite Time Series model achieved state-of-the-art performance in zero/few-shot forecasting, outperforming models ten times larger.</cite>
Safety and Guardrails
<cite index="12-3,12-4,12-5,12-6">Alongside the 2B and 8B language models, IBM released the Granite Guardian 3.0 model, which acts as a guardrail for inputs and outputs of other Granite 3.0 models—monitoring inputs for jailbreaking attacks and harmful prompts, and monitoring LLM outputs for bias, fairness, violence, and hallucinations.</cite>
Licensing and Transparency
<cite index="1-4">Consistent with the company's commitment to open-source AI, the Granite models are released under the permissive Apache 2.0 license.</cite> <cite index="2-3">In a departure from common industry practice for open models, IBM is providing a detailed disclosure of training data sets and methodologies in the Granite 3.0 technical paper.</cite> <cite index="9-5">IBM also open-sourced its Data Prep Kit and data curation recipes under the Apache 2.0 license, covering all data engineering tasks used in preparing training data.</cite>
Deployment and Enterprise Integration
<cite index="14-5">The models are available as an NVIDIA NIM microservice and accompanied by developer recipes in IBM's Granite Community on GitHub.</cite> <cite index="8-1">IBM also announced that Granite will become the default model of Consulting Advantage, an AI-powered delivery platform used by IBM's 160,000 consultants.</cite> <cite index="12-15">IBM Granite models are indemnified against legal issues related to training data when deployed within the IBM watsonx AI platform.</cite>
<cite index="1-7">By the end of 2024, the 3.0 8B and 2B language models were expected to include support for an extended 128,000-token context window and multimodal document understanding capabilities.</cite>