Text Generation
Transformers
Safetensors
hy_v3
hunyuan
hy3
Mixture of Experts
heretic
uncensored
decensored
abliterated
conversational
Instructions to use RadicalNotionAI/Hy3-heretic with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use RadicalNotionAI/Hy3-heretic with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="RadicalNotionAI/Hy3-heretic") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("RadicalNotionAI/Hy3-heretic") model = AutoModelForCausalLM.from_pretrained("RadicalNotionAI/Hy3-heretic", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use RadicalNotionAI/Hy3-heretic with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "RadicalNotionAI/Hy3-heretic" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RadicalNotionAI/Hy3-heretic", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/RadicalNotionAI/Hy3-heretic
- SGLang
How to use RadicalNotionAI/Hy3-heretic with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "RadicalNotionAI/Hy3-heretic" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RadicalNotionAI/Hy3-heretic", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "RadicalNotionAI/Hy3-heretic" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RadicalNotionAI/Hy3-heretic", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use RadicalNotionAI/Hy3-heretic with Docker Model Runner:
docker model run hf.co/RadicalNotionAI/Hy3-heretic
Upload README_CN.md with huggingface_hub
Browse files- README_CN.md +227 -0
README_CN.md
ADDED
|
@@ -0,0 +1,227 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
<p align="left">
|
| 2 |
+
<a href="https://huggingface.co/tencent/Hy3">English</a> | 中文
|
| 3 |
+
</p>
|
| 4 |
+
<br>
|
| 5 |
+
|
| 6 |
+
<p align="center">
|
| 7 |
+
<img src="assets/logo-zh.png" width="400"/> <br>
|
| 8 |
+
</p>
|
| 9 |
+
|
| 10 |
+
<div align="center" style="line-height: 1;">
|
| 11 |
+
|
| 12 |
+
|
| 13 |
+
[](#许可证)
|
| 14 |
+
|
| 15 |
+
[](https://huggingface.co/tencent/Hy3)
|
| 16 |
+
|
| 17 |
+
[](https://modelscope.cn/models/Tencent-Hunyuan/Hy3)
|
| 18 |
+
|
| 19 |
+
[](https://cnb.cool/ai-models/tencent/Hy3)
|
| 20 |
+
|
| 21 |
+
[](https://ai.gitcode.com/tencent_hunyuan/Hy3)
|
| 22 |
+
|
| 23 |
+
</div>
|
| 24 |
+
|
| 25 |
+
<p align="center">
|
| 26 |
+
🖥️ <a href="https://aistudio.tencent.com/"><b>官方网站</b></a> |
|
| 27 |
+
💬 <a href="https://github.com/Tencent-Hunyuan/Hy3"><b>GitHub</b></a></p>
|
| 28 |
+
|
| 29 |
+
---
|
| 30 |
+
|
| 31 |
+
## 目录
|
| 32 |
+
|
| 33 |
+
- [模型介绍](#模型介绍)
|
| 34 |
+
- [更强大的智能体能力](#更强大的智能体能力)
|
| 35 |
+
- [更可靠的产品体验](#更可靠的产品体验)
|
| 36 |
+
- [Benchmark 附录](#benchmark-附录)
|
| 37 |
+
- [新闻](#新闻)
|
| 38 |
+
- [模型链接](#模型链接)
|
| 39 |
+
- [快速开始](#快速开始)
|
| 40 |
+
- [推理和部署](#推理和部署)
|
| 41 |
+
- [vLLM](#使用-vllm-推理)
|
| 42 |
+
- [SGLang](#使用-sglang-推理)
|
| 43 |
+
- [模型微调](#模型微调)
|
| 44 |
+
- [量化工具](#量化工具)
|
| 45 |
+
- [许可证](#许可证)
|
| 46 |
+
- [联系我们](#联系我们)
|
| 47 |
+
|
| 48 |
+
---
|
| 49 |
+
|
| 50 |
+
## 模型介绍
|
| 51 |
+
|
| 52 |
+
**Hy3** 是由腾讯混元团队研发的快慢思考融合的混合专家模型,总参数量 295B,激活参数 21B,MTP 层参数 3.8B。4 月底发布 Hy3 Preview 后,我们在 50 多个业务中获得了广泛的反馈,修复了各种体验问题,进一步提升了后训练的质量和规模。今天,我们正式发布 Hy3。它展现出显著强于同尺寸模型的智能水平,并比肩更大尺寸旗舰模型的效果,大幅提升了在各类产品和生产力任务中的实用价值。
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
| 属性 | 值 |
|
| 56 |
+
|:---|:---|
|
| 57 |
+
| 架构 | 混合专家(MoE) |
|
| 58 |
+
| 总参数量 | 295B |
|
| 59 |
+
| 激活参数量 | 21B |
|
| 60 |
+
| MTP层参数量 | 3.8B |
|
| 61 |
+
| 层数(不含MTP层) | 80 |
|
| 62 |
+
| MTP层数 | 1 |
|
| 63 |
+
| 注意力头 | 64(GQA,8 个 KV 头,head dim 128) |
|
| 64 |
+
| 隐藏层维度 | 4096 |
|
| 65 |
+
| FFN 中间层维度 | 13312 |
|
| 66 |
+
| 上下文长度 | 256K |
|
| 67 |
+
| 词表大小 | 120832 |
|
| 68 |
+
| 专家数量 | 192 个专家,top-8 激活 |
|
| 69 |
+
| 支持精度 | BF16 |
|
| 70 |
+
|
| 71 |
+
## 更强大的智能体能力
|
| 72 |
+
|
| 73 |
+
Hy3 基于 Preview 进一步提升了后训练数据的质量和多样性,扩大了 RL 算力规模,在推理、智能体、长上下文等任务上显著进步,取得了比肩国内外更大尺寸旗舰模型(参数规模往往是 Hy3 的 2~5 倍)的效果。
|
| 74 |
+
|
| 75 |
+
<p align="center">
|
| 76 |
+
<img src="assets/benchmark.png" width="100%"/>
|
| 77 |
+
</p>
|
| 78 |
+
|
| 79 |
+
Hy3 在软件开发、办公生产、金融建模、前端设计、游戏制作等生产力任务上的进步尤其显著,可以成为高性价比的可靠选择。在内部组织的 270 位专家基于真实工作的模型盲测中,Hy3(均分 2.67 / 4)展现出优于 GLM5.1(均分 2.51 / 4)的表现,尤其在前端、数据与存储、CI/CD 等类别优势显著。
|
| 80 |
+
|
| 81 |
+
## 更可靠的产品体验
|
| 82 |
+
|
| 83 |
+
模型的实用体验不完全与榜单成绩挂钩。基于广泛的用户反馈和分析,我们定位并优化了一系列体验向能力,获得了产品侧一致且积极的评价。
|
| 84 |
+
|
| 85 |
+
**输出格式和工具调用稳定性**:我们显著改善了一系列基础底线问题,确保模型在各种工具设置和输出要求下达到生产级标准,工具调用的错误恢复能力和效率大幅提升。另外,Hy3 还增强了跨脚手架泛化性,不同脚手架(如 CodeBuddy、Cline、KiloCode)在 SWE Bench Verified 上的分数标准差控制在 4 个百分点以内。
|
| 86 |
+
|
| 87 |
+
**知识常识和抗幻觉能力**:基于“有依据才回答、无依据明示缺失,多来源信息不乱拼,数据和状态不乱编”的理想态,我们进行了细粒度的数据清洗和训练约束。在基于真实产品的内部评测中,Hy3 的幻觉率从 12.5% 降至 5.4%,常识错误率从 25.4% 降至 12.7%,显著改善了“张冠李戴”、无中生有、逻辑矛盾等问题。
|
| 88 |
+
|
| 89 |
+
**复杂上下文承接与多轮意图保持能力**:Hy3 在 SFT 与 RL 阶段联合优化了指代消解、省略还原及多轮约束继承等业务痛点问题,内部评测的多轮问题率从 17.4% 降至 7.9%。同时 Hy3 在长对话理解基准中取得显著跨越(如 MRCR 从 42.9% 升至 75.1%),输出更精炼的同时确保复杂意图在长程交互中不衰减、不跑偏。
|
| 90 |
+
|
| 91 |
+
## Benchmark 附录
|
| 92 |
+
|
| 93 |
+
<p align="center">
|
| 94 |
+
<img src="assets/benchmark-appendix.png" width="100%"/>
|
| 95 |
+
</p>
|
| 96 |
+
|
| 97 |
+
## 新闻
|
| 98 |
+
|
| 99 |
+
* 🔥 我们在 [Hugging Face](https://huggingface.co/tencent/Hy3)、[ModelScope](https://modelscope.cn/models/Tencent-Hunyuan/Hy3)、[GitCode](https://ai.gitcode.com/tencent_hunyuan/Hy3) 和 [CNB](https://cnb.cool/ai-models/tencent/Hy3) 开源了 **Hy3** 和 **Hy3-FP8** 模型权重。
|
| 100 |
+
|
| 101 |
+
## 模型链接
|
| 102 |
+
|
| 103 |
+
|
| 104 |
+
| 模型名 | 简介 | Hugging Face | ModelScope | GitCode | CNB |
|
| 105 |
+
|:---|:---|:---:|:---:|:---:|:---:|
|
| 106 |
+
| Hy3 | Instruct 模型 | 🤗 [Model](https://huggingface.co/tencent/Hy3) | [Model](https://modelscope.cn/models/Tencent-Hunyuan/Hy3) | [Model](https://ai.gitcode.com/tencent_hunyuan/Hy3) | [Model](https://cnb.cool/ai-models/tencent/Hy3) |
|
| 107 |
+
| Hy3-FP8 | FP8 量化 Instruct 模型 | 🤗 [Model](https://huggingface.co/tencent/Hy3-FP8) | [Model](https://modelscope.cn/models/Tencent-Hunyuan/Hy3-FP8) | [Model](https://ai.gitcode.com/tencent_hunyuan/Hy3-FP8) | [Model](https://cnb.cool/ai-models/tencent/Hy3-FP8) |
|
| 108 |
+
|
| 109 |
+
## 快速开始
|
| 110 |
+
|
| 111 |
+
建议先通过 [vLLM](#使用-vllm-推理) 或 [SGLang](#使用-sglang-推理) 部署服务,然后通过 OpenAI 兼容 API 调用:
|
| 112 |
+
|
| 113 |
+
```python
|
| 114 |
+
from openai import OpenAI
|
| 115 |
+
|
| 116 |
+
client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="EMPTY")
|
| 117 |
+
|
| 118 |
+
response = client.chat.completions.create(
|
| 119 |
+
model="hy3",
|
| 120 |
+
messages=[
|
| 121 |
+
{"role": "user", "content": "你好!请简单介绍一下你自己。"},
|
| 122 |
+
],
|
| 123 |
+
temperature=0.9,
|
| 124 |
+
top_p=1.0,
|
| 125 |
+
# reasoning_effort: "no_think"(默认,直接回复)、"low"、"high"(深度思维链)
|
| 126 |
+
extra_body={"chat_template_kwargs": {"reasoning_effort": "no_think"}},
|
| 127 |
+
)
|
| 128 |
+
print(response.choices[0].message.content)
|
| 129 |
+
```
|
| 130 |
+
|
| 131 |
+
> **推荐参数**:`temperature=0.9`,`top_p=1.0`。
|
| 132 |
+
>
|
| 133 |
+
> **推理模式**:复杂任务(数学、编程、推理)建议设置 `reasoning_effort="high"`,日常对话可使用默认的 `"no_think"` 直接回复。
|
| 134 |
+
|
| 135 |
+
具体部署方式请参考下方[推理和部署](#推理和部署)章节。
|
| 136 |
+
|
| 137 |
+
## 推理和部署
|
| 138 |
+
|
| 139 |
+
Hy3 总参数量为 295B,当使用 8 张 GPU 时,建议使用 H20-3e 或其他有更大显存的卡型。
|
| 140 |
+
|
| 141 |
+
对于生产环境部署,我们建议使用 vLLM 或 SGLang,这两个框架都为 Hy3 提供了专门的配置方案:
|
| 142 |
+
|
| 143 |
+
- [vLLM](https://github.com/vllm-project/vllm) - 请查阅 [vLLM recipes](https://recipes.vllm.ai/tencent/Hy3)
|
| 144 |
+
|
| 145 |
+
- [SGLang](https://docs.sglang.io/) - 请查阅 [SGLang cookbook](https://lmsysorg.mintlify.app/cookbook/autoregressive/Tencent/Hy3)
|
| 146 |
+
|
| 147 |
+
### vLLM
|
| 148 |
+
|
| 149 |
+
从源码构建 vLLM:
|
| 150 |
+
|
| 151 |
+
```bash
|
| 152 |
+
uv venv --python 3.12 --seed --managed-python
|
| 153 |
+
source .venv/bin/activate
|
| 154 |
+
git clone https://github.com/vllm-project/vllm.git
|
| 155 |
+
cd vllm
|
| 156 |
+
uv pip install --editable . --torch-backend=auto
|
| 157 |
+
```
|
| 158 |
+
|
| 159 |
+
启动 vLLM 服务,开启 MTP:
|
| 160 |
+
|
| 161 |
+
```bash
|
| 162 |
+
# Switch to trtllm backend to work-around mnnvl workspace size issue.
|
| 163 |
+
export VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm
|
| 164 |
+
|
| 165 |
+
vllm serve tencent/Hy3 \
|
| 166 |
+
--tensor-parallel-size 8 \
|
| 167 |
+
--speculative-config.method mtp \
|
| 168 |
+
--speculative-config.num_speculative_tokens 2 \
|
| 169 |
+
--tool-call-parser hy_v3 \
|
| 170 |
+
--reasoning-parser hy_v3 \
|
| 171 |
+
--enable-auto-tool-choice \
|
| 172 |
+
--port 8000 \
|
| 173 |
+
--served-model-name hy3
|
| 174 |
+
```
|
| 175 |
+
|
| 176 |
+
### SGLang
|
| 177 |
+
|
| 178 |
+
从源码构建 SGLang:
|
| 179 |
+
|
| 180 |
+
```bash
|
| 181 |
+
git clone https://github.com/sgl-project/sglang
|
| 182 |
+
cd sglang
|
| 183 |
+
pip3 install pip --upgrade
|
| 184 |
+
pip3 install "transformers>=5.6.0"
|
| 185 |
+
pip3 install -e "python"
|
| 186 |
+
```
|
| 187 |
+
|
| 188 |
+
启动 SGLang 服务,开启 MTP:
|
| 189 |
+
|
| 190 |
+
```bash
|
| 191 |
+
python3 -m sglang.launch_server \
|
| 192 |
+
--model tencent/Hy3 \
|
| 193 |
+
--tp-size 8 \
|
| 194 |
+
--tool-call-parser hunyuan \
|
| 195 |
+
--reasoning-parser hunyuan \
|
| 196 |
+
--speculative-num-steps 2 \
|
| 197 |
+
--speculative-eagle-topk 1 \
|
| 198 |
+
--speculative-num-draft-tokens 3 \
|
| 199 |
+
--speculative-algorithm EAGLE \
|
| 200 |
+
--port 8000 \
|
| 201 |
+
--served-model-name hy3
|
| 202 |
+
```
|
| 203 |
+
|
| 204 |
+
## 模型微调
|
| 205 |
+
|
| 206 |
+
Hy3 提供了完整的模型微调流程,详细的微调文档请参考:[模型微调指南](https://huggingface.co/tencent/Hy3/blob/main/finetune/README_CN.md)
|
| 207 |
+
|
| 208 |
+
## 量化工具
|
| 209 |
+
|
| 210 |
+
我们提供了 [AngelSlim](https://github.com/tencent/AngelSlim)——一套易用、全面、高效的大模型压缩工具包,涵盖常用量化算法、低比特量化和投机采样等能力。
|
| 211 |
+
|
| 212 |
+
## 许可证
|
| 213 |
+
|
| 214 |
+
|
| 215 |
+
Hy3 基于 **Apache 2.0 许可证** 发布。详情请参阅 [LICENSE](https://huggingface.co/tencent/Hy3/blob/main/LICENSE)。
|
| 216 |
+
|
| 217 |
+
## 联系我们
|
| 218 |
+
|
| 219 |
+
如有问题或建议,欢迎通过邮件联系我们:
|
| 220 |
+
|
| 221 |
+
📧 **hunyuan_opensource@tencent.com**
|
| 222 |
+
|
| 223 |
+
---
|
| 224 |
+
|
| 225 |
+
<p align="center">
|
| 226 |
+
<i>Hy3 由腾讯混元团队研发。</i>
|
| 227 |
+
</p>
|