Instructions to use AXERA-TECH/DeepSeek-R1-Distill-Qwen-7B-GPTQ-Int4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AXERA-TECH/DeepSeek-R1-Distill-Qwen-7B-GPTQ-Int4 with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("AXERA-TECH/DeepSeek-R1-Distill-Qwen-7B-GPTQ-Int4", dtype="auto") - Notebooks
- Google Colab
- Kaggle
File size: 277 Bytes
549e2e3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 | {
"enable_temperature" : true,
"temperature" : 0.9,
"enable_repetition_penalty" : false,
"repetition_penalty" : 1.2,
"penalty_window" : 20,
"enable_top_p_sampling" : false,
"top_p" : 0.8,
"enable_top_k_sampling" : true,
"top_k" : 10
} |