This document mainly conducts research and trial on the latest Granite 3.0 launched by IBM, forming a research report and guidance on the usage process, making it convenient for readers to understand and trial the latest Granite 3.0 model and learn to find resources related to Granite 3.0.
The purpose of writing this document is to record the work content and usage methods of researching Granite 3.0, as well as the organization of relevant open source resources and documents involved in the process.
By following the guidance of this document, you can quickly and effectively understand the Granite 3.0 model. Through the guidance of this document, you can quickly deploy and use the Granite 3.0 model locally. Consider it a reference for Granite 3.0 related work.
Total text: 4826 words
Estimated reading time: 14-20 minutes
01 · Introduction to Granite3.0 model
1 · Contents of the Granite 3.0 model
IBM’s Granite 3.0 family includes:
- General/Language Model:
-
Granite 3.0 8B Instruct,
-
Granite 3.0 2B Instruct,
-
Granite 3.0 8B Base,
-
Granite 3.0 2B Base
- Guardrails and safety models:
-
Granite Guardian 3.0 8B,
-
Granite Guardian 3.0 2B
- Expert mixture model:
-
Granite 3.0 3B-A800M Instruct,
-
Granite 3.0 1B-A400M Instruct,
-
Granite 3.0 3B-A800M Base,
-
Granite 3.0 1B-A400M Base
General/language models are mainly used for natural language understanding and generation tasks, and are suitable for a wide range of scenarios, such as text generation, question and answer systems, dialogue systems, etc.
1. Granite 3.0 8B Instruct is a large language model with 8 billion parameters. It is command-tuned to handle complex tasks such as generating detailed answers, writing articles, translations, and more.
2. Granite 3.0 2B Instruct is a medium-sized language model with 2 billion parameters. It has also been optimized by instructions and is suitable for medium-complexity tasks, such as simple text generation, question and answer, etc.
3. Granite 3.0 8B Base is a large basic language model with 8 billion parameters. It is not tuned for a specific task and is suitable for general tasks requiring a large number of parameters.
4. Granite 3.0 2B Base is a medium-sized basic language model with 2 billion parameters. It is also not tuned for specific tasks and is suitable for medium-sized general tasks.
护栏与安全模型是专门设计用于提高模型的安全性和可靠性，防止生成有害或不当的内容。
1. Granite Guardian 3.0 8B is a large security model with 8 billion parameters. It is specifically trained to identify and filter harmful content, ensuring the content generated is safe and appropriate.
2. Granite Guardian 3.0 2B is a medium-sized security model with 2 billion parameters. It has also undergone specialized training and is suitable for scenarios requiring medium-scale security protection.
The expert mixture model combines expertise from different fields and is suitable for tasks in specific fields, such as finance, medical, legal, etc.
1. Granite 3.0 3B-A800M Instruct is a medium-sized hybrid model with a parameter volume of 3 billion and an additional professional knowledge module of 800 million parameters. After instruction tuning, it is suitable for tasks that require professional knowledge, such as financial analysis, legal consulting, etc.
2. Granite 3.0 1B-A400M Instruct is a smaller-scale hybrid model with 1 billion parameters and an additional professional knowledge module of 400 million parameters. Also command-tuned for tasks requiring moderate expertise.
3. Granite 3.0 3B-A800M Base is a medium-sized basic hybrid model with a parameter volume of 3 billion and an additional professional knowledge module of 800 million parameters. Not tuned for specific tasks, suitable for general tasks that require a large number of parameters and expertise.
4. Granite 3.0 1B-A400M Base is a smaller-scale basic hybrid model with 1 billion parameters and an additional professional knowledge module with 400 million parameters. It is also not tuned for specific tasks and is suitable for general tasks requiring moderate parameters and expertise.
2 · Advantages of Granite 3.0 model
-
raw performance
The Granite 3.0 language model shows good results in raw performance.
On standard academic benchmarks defined by Hugging Face's OpenLLM leaderboard, the overall performance of the Granite 3.0 8B Instruct model is on average ahead of the state-of-the-art performance of similarly sized open source models in Meta and Mistral. In IBM's advanced AttaQ security benchmark, the Granite 3.0 8B Instruct model outperformed Meta and Mistral's models in all measured security dimensions.
On average, the Granite 3.0 8B Instruct model outperforms similarly sized open source models in Mistral and Meta on core enterprise tasks in the areas of Retrieval Augmented Generation (RAG), tool usage, and cybersecurity.
-
Training data set
Granite 3.0 models are trained on more than 12 trillion tokens from 12 different natural languages and 116 different programming languages using a novel two-stage training approach that leverages the results of thousands of experiments designed to optimize data quality, data selection, and training parameters. By the end of the year, 3.0 8B and 2B language models are expected to support expanded 128K context windows and multimodal document understanding capabilities.
-
Reasoning costs
Demonstrating the perfect balance of performance and inference cost, IBM offers its Granite Mix of Experts (MoE) agency models, Granite 3.0 1B-A400M and Granite 3.0 3B-A800M, which are smaller and lightweight models that can be used for low-latency applications as well as CPU-based deployments.
IBM also announced an updated version of its pre-trained Granite time series model, with an earlier version released earlier this year. Trained on 3x the data, these new models outperformed on all three major time series benchmarks, outperforming 10x larger models from Google, Alibaba, and others. The updated model also provides greater modeling flexibility, supporting external variables and rolling forecasts.
02 · Granite3.0 model trial
1 · How to use the Granite 3.0 model
1. Open your web browser and navigate to the Hugging Face website (domestic Chinese mirror website link: https://hf-mirror.com/ ).
2. Install the Hugging Face Transformers Library, which provides the use of LLMs.
3. In the search bar at the top of the page, type "Granite 3.0" and press Enter. This will display a list of available Granite 3.0 models.
4. Select the Granite 3.0 model to download.
5. Click a model's name to go to its page.
Here I choose Granite 3.0 8B Instruct for trial.
6. Click Use this model and download to the appropriate library.
Download the parameters of the officially trained model. (Model parameter address: https://hf-mirror.com/ibm-granite/granite-3.0-8b-instruct/tree/main)
In addition, the languages supported by Granite 3.0 are: English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch and Chinese. Users can fine-tune Granite 3.0 models for languages other than these 12.
- Run example 1:Please list one IBM Research laboratory located in the United States. You should only output its name and location.
Answer:IBM Almaden Research Center - San Jose, California
- Run example 2:What travel destinations do you recommend during the Chinese New Year?
Answer:The Chinese New Year is the most important traditional festival in China, and there are unique travel destination recommendations in various places. For example, the Temple of Heaven in Beijing, the Great Wall in Shanghai, and the mosques in Guangzhou are all very popular tourist attractions.
7. Select a format that is compatible with your development environment.
8. Save the file to your computer.
03·Potential assistance of Granite3.0 model
1 · Granite 3.0 model helps enterprises
Universal language model:
The Granite 3.0 language model provides language models of different sizes, including a large language model with 8 billion parameters and a medium-sized language model with 2 billion parameters. Enterprises can choose large or medium-sized models for practical use based on actual business needs.
In addition, Granite 3.0 8B Base has 8 billion parameters and has not been tuned for specific tasks. It is suitable for large enterprises or enterprises with special needs to deeply customize and develop applications that meet their own business characteristics. Compared with 8B Base, Granite 3.0 2B Base has a lower cost and is more suitable for companies with limited budgets. Although it has fewer parameters, it still performs well on common text processing tasks and can meet the basic needs of most companies.
Guardrails and Safety Models:
Guardrails and security models not only protect enterprises from potential security threats, but also ensure that enterprises comply with relevant laws and regulations when using advanced technologies and data processing. Using guardrails and security models can provide core encryption, access control and auditing functions, providing comprehensive security protection and compliance support. In summary, the security model of Granite 3.0 can provide a strong guarantee for the data security of enterprises, and high-value charging products can be further developed based on the security model of Granite 3.0.
Expert mixture model:
Mixture of Experts (MoE) is a technique that improves overall model performance by combining multiple expert models. This model performs well when processing large-scale data and complex tasks, and can significantly improve prediction accuracy and computational efficiency.
The expert mixture model is suitable for tasks that require professional knowledge, such as financial analysis, legal consulting, etc., and can be trained with professional data tailored to the business needs of specific enterprise users.
2 · Deployment steps for enterprises implementing the Granite 3.0 model
(1) Clarify the needs:
Determine the specific problems or goals that the enterprise hopes to solve by deploying Granite 3.0, such as improving data processing efficiency, enhancing security, optimizing customer experience, etc. Evaluate existing hardware and software resources to determine whether additional investments or upgrades are needed.
(2) Choose the appropriate version
Based on the above requirements, select a suitable model from the general language model, guardrail and safety model, and expert hybrid model, and select the appropriate model parameters based on the actual data scale.
(3) Hardware resources and environment preparation
Enterprises need to ensure that they have sufficient computing resources (such as CPU, GPU, memory) to run the Granite 3.0 model. Ensure network bandwidth and stability, especially for distributed deployments and remote access situations.
Make sure the operating system the server is running is compatible with the requirements of Granite 3.0. Install necessary dependent libraries and tools, such as Python, TensorFlow, etc.
(4) Model deployment and integration
Download selected Granite 3.0 models from Hugging Face official channels. Install and configure the model according to the official documentation, and use business-corresponding data to complete training or fine-tuning of the model. Connect Granite 3.0’s API with existing business systems to ensure smooth data transmission.
(5) Testing and verification
After the integration is completed, each functional module needs to be unit tested to ensure that it works as expected. Test the performance indicators of the model, such as response time, throughput, etc. Conduct end-to-end integration testing to ensure that various modules work together and there are no compatibility issues.
04 · Comparison between Granite3.0 model and other models 1 · Model architecture
-
Granite 3.0
Architecture type: Granite 3.0 is a large-scale language model based on Transformer. The Transformer architecture effectively handles long dependencies through self-attention, allowing the model to perform well when processing long texts.
Number of parameters: Granite 3.0 has multiple versions, among which Granite-3.0-8B has 8 billion parameters and is a very large model. In addition, there is a dense version of Granite-3.0, which improves performance by optimizing the model structure and training methods.
Optimization technology: Granite 3.0 introduces a variety of optimization technologies, such as sparse attention mechanism, mixed precision training, etc., to improve training efficiency and model performance.
Other models:
· BERT：Transformer-based bidirectional encoder representation, mainly used for natural language understanding tasks. The number of parameters is usually small. For example, BERT-base has 110 million parameters and BERT-large has 340 million parameters.
· GPT：Autoregressive language model based on Transformer, mainly used for text generation tasks. With 175 billion parameters, GPT-3 is one of the largest public language models currently available.
· RoBERTa：An improved version of BERT that improves performance with more training data and longer training time.
· T5：The pre-trained model based on Transformer uses prefix conditioning technology to support a variety of natural language processing tasks.
2 · Training data
-
Granite 3.0
Data volume: Granite 3.0 uses a large amount of high-quality training data, covering a wide range of fields and topics. High-quality data helps the model learn richer language patterns and knowledge.
Data diversity: Training data includes text, code, dialogue and other types, allowing the model to perform well in a variety of tasks.
Other models
· BERT：The training data mainly comes from Wikipedia and BookCorpus, with a total of 16GB of text data.
· GPT：The training data comes from the Internet, including web pages, books, news and other sources, and the amount of data is larger.
· RoBERTa：A larger training data set was used, including Common Crawl and other sources, with a data size of approximately 160GB.
· T5：The training data comes from the C4 data set, which is a clean data set from Common Crawl and the data volume is also very large.
3·Application scenarios
-
Granite 3.0 is suitable for:
1. Content creation: suitable for tasks such as article generation and creative writing.
2. Intelligent customer service: suitable for building dialogue systems and intelligent customer service.
3. Machine translation: suitable for cross-language text translation.
4. Sentiment analysis: suitable for social media monitoring, user comment analysis, etc.
Other models
· BERT：Suitable for natural language understanding tasks, such as sentiment analysis, named entity recognition, etc.
· GPT：Suitable for text generation tasks, such as article generation, creative writing, etc.
· RoBERTa：Suitable for natural language understanding tasks, especially those requiring high accuracy.
· T5：Suitable for a variety of natural language processing tasks, especially in scenarios that require multi-task learning.
4 · Summary of Granite 3.0 model analysis
Granite-3.0 models, both the 8B version and the dense version, demonstrate superior performance across multiple tasks and domains. Especially version 8B, with its large-scale parameters and high-quality training data, is able to achieve significant advantages in complex tasks. The dense version improves the efficiency and performance of the model by optimizing the structure and training methods under the same parameter amount, and is suitable for application scenarios that require high performance and fast response.
The Granite-3.0 model surpasses the baseline model in many aspects, providing more powerful tools and support for natural language processing tasks.
The Granite 3.0 model demonstrates its superiority in multiple aspects, especially in terms of large-scale parameters, high-quality training data, optimization techniques, and multi-task performance. Compared with traditional models, Granite 3.0 shows greater adaptability and higher performance in complex tasks and diverse data. However, this also brings higher computing resource requirements.
References and resource links
[1] IBM launches Granite 3.0: a high-performance AI model specially built for enterprises. Article link: https://china.newsroom.ibm.com/2024-10-22-IBM-Granite-3-0-AI
[2] IBM Granite 3.0 experience. Article link: https://www.ibm.com/granite
[3] Huggingface’s domestic official mirror website. Resource link: https://hf-mirror.com/
[4] IBM Granite 3.0 model parameter download address after official training is completed. Resource link:
https://hf-mirror.com/ibm-granite/granite-3.0-8b-instruct/tree/main
[5] IBM Granite 3.0 model usage tutorial. Resource link:
https://hf-mirror.com/ibm-granite/granite-3.0-8b-instruct
[6] Granite 3.0 paper link. Resource link:
https://github.com/ibm-granite/granite-3.0-language-models/blob/main/paper.pdf
Machine-translated from the Chinese original IBM Granite3.0 调研报告 (published 2024-11-28). WeChat original ↗