> ## Content Index > Fetch the complete content index at: https://www.hubwiz.com/blog/llms.txt > Use this file to discover other available public pages before exploring further. # IBM Granite 4.0-Nano边缘AI模型 - URL: https://www.hubwiz.com/blog/ibm-granite-4-0-nano-edge-ai-model/ - Published: 2025-11-04T09:14:41.000Z - Updated: 2025-11-04T09:14:41.000Z - Description: IBM推出了Granite 4.0 Nano模型系列，致力于创建强大且实用的大语言模型（LLM），这些模型特别优化用于边缘和设备端应用。 - Author: admin - Tags: MODEL-ZOO, GRANITE 4.0-NANO, #Import 2026-10-14 14:41 IBM推出了**Granite 4.0 Nano**模型系列，致力于创建强大且实用的大语言模型（LLM），这些模型特别优化用于**边缘和设备端应用**。这些模型的参数量从大约3.5亿到15亿不等，在通用知识、数学、代码和安全性等标准基准测试中，与竞争对手类似规模的模型相比，其能力显著增强。 ## 1、概述 该发布包括四个主要变体，包括基于**新型高效混合-SSM架构**（如Granite 4.0 H 1B和H 350M）的模型，以及传统的变压器版本，以确保与各种运行时的兼容性（如`llama.cpp`）。至关重要的是，Granite 4.0 Nano模型建立在与更大的Granite 4.0家族相同的强大训练流程和超过15万亿个标记的数据基础上。 为了广泛适用性和信心，所有Nano模型都采用**Apache 2.0许可证**发布，并带有**IBM的ISO 42001认证**，确保负责任的模型开发和治理，使用户可以放心部署，符合全球标准。 ![](https://www.hubwiz.com/blog/content/images/2025/11/image-17.png) ## 2、如何尝试Granite 4 Nano？ 这些最先进的模型可以通过两个主要渠道方便地访问：简化部署平台[Ollama](https://www.ibm.com/granite/docs/run/granite-with-ollama-mac?ref=hubwiz.com)，以及在Hugging Face上维护的全面存储库。这两个资源的直接访问链接已汇总在专用的“链接”部分，以便于您的使用。为了清楚地说明集成的非凡简便性和速度，我已改编并增强了基础代码示例，如下一节所示。 - 准备环境 ``` python3 -m venv venv source venv/bin/activate pip install --upgrade pip ``` - 安装所需的必要包 📦 ``` # requirements.txt HuggingFace torch transformers accelerate torchvision torchaudio ``` ``` pip install -r requirements.txt ``` - 一旦环境准备就绪，只需复制并运行这两个简单的应用程序！ > 第一个代码使用tokenizer的模板简单构建一个聊天提示，以在用户询问波士顿天气时启用工具调用行为。在对准备好的输入进行分词并将其移动到选定的设备后，模型会生成输出序列。 第二个脚本使用预训练的LLM执行完整的机器学习推理管道。然后加载IBM Granite 4.0（3.5亿参数）模型和分词器，为研究实验室位置查询准备用户提示，并从模型生成响应。 ``` import torch from transformers import AutoModelForCausalLM, AutoTokenizer import json import os # Import os for file system operations # --- Device Detection and Selection --- # Automatically determine the best device available (CUDA > MPS > CPU) if torch.cuda.is_available(): device = "cuda" elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available(): # MPS (Metal Performance Shaders) is the accelerator for Apple Silicon (M1/M2/M3) device = "mps" else: device = "cpu" print(f"Selected device: {device}") # --- End Device Detection --- model_path = "ibm-granite/granite-4.0-350M" tokenizer = AutoTokenizer.from_pretrained(model_path) # Pass the automatically determined device to device_map # The model will now load onto the CPU (or MPS/CUDA if available) model = AutoModelForCausalLM.from_pretrained(model_path, device_map=device) model.eval() tools = [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather for a specified city.", "parameters": { "type": "object", "properties": { "city": { "type": "string", "description": "Name of the city" } }, "required": ["city"] } } } ] # change input text as desired chat = [ { "role": "user", "content": "What's the weather like in Boston right now?" }, ] chat = tokenizer.apply_chat_template(chat, \ tokenize=False, \ tools=tools, \ add_generation_prompt=True) # tokenize the text and move to the selected device input_tokens = tokenizer(chat, return_tensors="pt").to(device) # generate output tokens output = model.generate(**input_tokens, max_new_tokens=100) # decode output tokens into text output = tokenizer.batch_decode(output) # --- Save output to file in Markdown format --- output_dir = "./output" output_file = os.path.join(output_dir, "output.md") # Create the output directory if it doesn't exist. # The exist_ok=True argument prevents an error if the directory already exists. try: os.makedirs(output_dir, exist_ok=True) # Write the output to the Markdown file with open(output_file, "w", encoding="utf-8") as f: # The output from batch_decode is a list, we take the first item (the generated text) f.write(output[0]) # Confirmation message print(f"\nModel output saved successfully to {output_file}") # Optionally print the content to the console for immediate review print("\n--- Generated Content ---\n") print(output[0]) print("\n-------------------------") except Exception as e: print(f"\nAn error occurred while trying to save the output file: {e}") # --- End File Saving Logic --- ``` ``` import torch from transformers import AutoModelForCausalLM, AutoTokenizer import os # Import os for file system operations # --- Device Detection and Selection --- # Automatically determine the best device available (CUDA > MPS > CPU) if torch.cuda.is_available(): device = "cuda" elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available(): # MPS (Metal Performance Shaders) is the accelerator for Apple Silicon (M1/M2/M3) device = "mps" else: device = "cpu" print(f"Selected device: {device}") # --- End Device Detection --- model_path = "ibm-granite/granite-4.0-350M" tokenizer = AutoTokenizer.from_pretrained(model_path) # Pass the automatically determined device to device_map model = AutoModelForCausalLM.from_pretrained(model_path, device_map=device) model.eval() # change input text as desired chat = [ { "role": "user", "content": "Please list one IBM Research laboratory located in the United States. You should only output its name and location." }, ] chat = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True) # tokenize the text and move to the selected device input_tokens = tokenizer(chat, return_tensors="pt").to(device) # generate output tokens output = model.generate(**input_tokens, max_new_tokens=100) # decode output tokens into text output = tokenizer.batch_decode(output) # --- Save output to file in Markdown format --- output_dir = "./output" output_file = os.path.join(output_dir, "output.md") # Create the output directory if it doesn't exist. try: os.makedirs(output_dir, exist_ok=True) # Write the output to the Markdown file with open(output_file, "w", encoding="utf-8") as f: # The output from batch_decode is a list, we take the first item (the generated text) f.write(output[0]) # Confirmation message print(f"\nModel output saved successfully to {output_file}") # Optionally print the content to the console for immediate review print("\n--- Generated Content ---\n") print(output[0]) print("\n-------------------------") except Exception as e: print(f"\nAn error occurred while trying to save the output file: {e}") # --- End File Saving Logic --- ``` 您将获得这些输出 📄 ``` <|start_of_role|>system<|end_of_role|>You are a helpful assistant with access to the following tools. You may call one or more tools to assist with the user query. You are provided with function signatures within XML tags: {"type": "function", "function": {"name": "get_current_weather", "description": "Get the current weather for a specified city.", "parameters": {"type": "object", "properties": {"city": {"type": "string", "description": "Name of the city"}}, "required": ["city"]}}} For each tool call, return a json object with function name and arguments within XML tags: {"name": , "arguments": } . If a tool does not exist in the provided list of tools, notify the user that you do not have the ability to fulfill the request.<|end_of_text|> <|start_of_role|>user<|end_of_role|>What's the weather like in Boston right now?<|end_of_text|> <|start_of_role|>assistant<|end_of_role|> {"name": "get_current_weather", "arguments": {"city": "Boston"}} <|end_of_text|> ====== <|start_of_role|>system<|end_of_role|>You are a helpful assistant. Please ensure responses are professional, accurate, and safe.<|end_of_text|> <|start_of_role|>user<|end_of_role|>Please list one IBM Research laboratory located in the United States. You should only output its name and location.<|end_of_text|> <|start_of_role|>assistant<|end_of_role|>IBM Research Laboratory: Cambridge Research Laboratory<|end_of_text|> ``` 就这样 🥇 ## 3、结束语 Granite 4.0模型系列代表了向高效企业级AI的重要转变，通过关注可访问性而非规模来重新定义性能。其关键优势在于创新的混合Mamba/Transformer架构，这显著降低了内存需求——通常超过70%——使得在**基本且经济的硬件**上进行强大的推理成为可能，包括消费级GPU和边缘设备。至关重要的是，作为在Apache 2.0许可下的开源产品，Granite 4.0模型赋予开发者完全的操作自主权，允许深度定制、本地部署以增强数据隐私，并实现完全透明。这种卓越的效率和开放治理的结合降低了进入门槛，使复杂的工作流程如RAG和函数调用民主化，同时确保了现实世界业务采用所需的控制和信任。 --- 原文链接：[IBM Granite 4.0-Nano](https://alain-airom.medium.com/ibm-granite-4-0-nano-b4a3a4d9cef7?ref=hubwiz.com) 汇智网翻译整理，转载请标明出处