Instructions to use Qwen/Qwen-7B-Chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Qwen/Qwen-7B-Chat with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Qwen/Qwen-7B-Chat", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen-7B-Chat", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Qwen/Qwen-7B-Chat with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Qwen/Qwen-7B-Chat" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Qwen/Qwen-7B-Chat", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Qwen/Qwen-7B-Chat
- SGLang
How to use Qwen/Qwen-7B-Chat with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Qwen/Qwen-7B-Chat" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Qwen/Qwen-7B-Chat", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Qwen/Qwen-7B-Chat" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Qwen/Qwen-7B-Chat", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Qwen/Qwen-7B-Chat with Docker Model Runner:
docker model run hf.co/Qwen/Qwen-7B-Chat
update react_prompt.md
Browse files- examples/react_prompt.md +3 -3
examples/react_prompt.md
CHANGED
|
@@ -11,7 +11,7 @@
|
|
| 11 |
假设我们有如下的一个适合用工具处理的 query,以及有夸克搜索、通义万相文生图这两个工具:
|
| 12 |
|
| 13 |
```py
|
| 14 |
-
query = '
|
| 15 |
|
| 16 |
TOOLS = [
|
| 17 |
{
|
|
@@ -123,7 +123,7 @@ Final Answer: the final answer to the original input question
|
|
| 123 |
|
| 124 |
Begin!
|
| 125 |
|
| 126 |
-
Question:
|
| 127 |
```
|
| 128 |
|
| 129 |
将这个 prompt 送入千问,并记得设置 "Observation" 为 stop word (见本文末尾的 FAQ)—— 即让千问在预测到要生成的下一个词是 "Observation" 时马上停止生成 —— 则千问在得到这个 prompt 后会生成如下的结果:
|
|
@@ -170,7 +170,7 @@ Final Answer: the final answer to the original input question
|
|
| 170 |
|
| 171 |
Begin!
|
| 172 |
|
| 173 |
-
Question:
|
| 174 |
Thought: 我应该使用通义万相API来生成一张五彩斑斓的黑的图片。
|
| 175 |
Action: image_gen
|
| 176 |
Action Input: {"query": "五彩斑斓的黑"}
|
|
|
|
| 11 |
假设我们有如下的一个适合用工具处理的 query,以及有夸克搜索、通义万相文生图这两个工具:
|
| 12 |
|
| 13 |
```py
|
| 14 |
+
query = '现在给我画个五彩斑斓的黑。'
|
| 15 |
|
| 16 |
TOOLS = [
|
| 17 |
{
|
|
|
|
| 123 |
|
| 124 |
Begin!
|
| 125 |
|
| 126 |
+
Question: 现在给我画个五彩斑斓的黑。
|
| 127 |
```
|
| 128 |
|
| 129 |
将这个 prompt 送入千问,并记得设置 "Observation" 为 stop word (见本文末尾的 FAQ)—— 即让千问在预测到要生成的下一个词是 "Observation" 时马上停止生成 —— 则千问在得到这个 prompt 后会生成如下的结果:
|
|
|
|
| 170 |
|
| 171 |
Begin!
|
| 172 |
|
| 173 |
+
Question: 现在给我画个五彩斑斓的黑。
|
| 174 |
Thought: 我应该使用通义万相API来生成一张五彩斑斓的黑的图片。
|
| 175 |
Action: image_gen
|
| 176 |
Action Input: {"query": "五彩斑斓的黑"}
|