Instructions to use empero-ai/Qwen3.8-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use empero-ai/Qwen3.8-4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="empero-ai/Qwen3.8-4B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("empero-ai/Qwen3.8-4B")
model = AutoModelForMultimodalLM.from_pretrained("empero-ai/Qwen3.8-4B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use empero-ai/Qwen3.8-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "empero-ai/Qwen3.8-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "empero-ai/Qwen3.8-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/empero-ai/Qwen3.8-4B

SGLang

How to use empero-ai/Qwen3.8-4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "empero-ai/Qwen3.8-4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "empero-ai/Qwen3.8-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "empero-ai/Qwen3.8-4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "empero-ai/Qwen3.8-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use empero-ai/Qwen3.8-4B with Docker Model Runner:
```
docker model run hf.co/empero-ai/Qwen3.8-4B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Qwen3.8-4B

Developed by Empero

This repository contains model weights and configuration files in the Hugging Face Transformers format.

These artifacts are compatible with Hugging Face Transformers, vLLM, SGLang, and other standard runtimes with Qwen3.5 architecture support.

Qwen3.8-4B is a full-parameter distillation of Qwen3.8 2.4T A95B into the Qwen3.5-4B architecture. The student was trained on ~45,000 curated teacher traces from our internal Qwen3.8 distillation datasets — dense chain-of-thought spanning mathematics, general reasoning, and instruction following, quality-filtered before training.

The objective: bring the reasoning behavior of a frontier-scale teacher into a 4B that runs comfortably on consumer hardware.

Highlights

Distilled chain-of-thought — every answer opens with a <think> block learned directly from Qwen3.8 2.4T A95B traces rather than synthetic self-generated reasoning.
4B weight class — bf16 fits in ~8 GB; quantized builds run on laptops and consumer GPUs.
Native function calling per Qwen3.5's specification — no wrapper or tool-specific fine-tune required.
262,144-token native context, inherited from the Qwen3.5 base.
Full fine-tune — every parameter updated; not an adapter.

Model Overview

Type: Causal Language Model (text path of a vision-language base)
Base: Qwen/Qwen3.5-4B
Number of Parameters: 4B
Training: SFT (off-policy distillation) on ~45,000 teacher traces
Teacher: Qwen3.8 2.4T A95B (internal distillation datasets)
Context Length: 262,144 natively

Benchmark Results

Measured with lm-evaluation-harness, HF backend, identical settings for base and student. Both models are reasoning models and are evaluated with the CoT protocols (gsm8k_cot, mmlu_flan_cot_zeroshot); MMLU covers all 57 subjects (~1,700 questions). Flexible-extract is the primary metric; strict-match requires exact answer formatting.

Task	Metric	Qwen3.5-4B (base)	Qwen3.8-4B	Δ
gsm8k_cot	exact_match (flexible)	0.850	0.785	−0.065
gsm8k_cot	exact_match (strict)	0.850	0.785	−0.065
mmlu (CoT, 57 subjects)	acc (flexible-extract)	0.354	0.553	+0.199
mmlu (CoT, 57 subjects)	acc (strict-match)	0.071	0.233	+0.162

Sampling for generation: temperature=0.6, top_p=0.95, top_k=20 (Qwen3.5 recommended settings).

Quickstart

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "empero-ai/Qwen3.8-4B"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": "A snail is at the bottom of a 10-meter well. Each day it climbs 3 meters, each night it slips back 2. How many days until it escapes?"}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)

out = model.generate(inputs, max_new_tokens=16384,
                     temperature=0.6, top_p=0.95, top_k=20, do_sample=True)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

A recent transformers release with Qwen3.5 support is required, along with the Gated DeltaNet kernels (flash-linear-attention and a CUDA-matched causal_conv1d build) — without them the linear-attention layers fall back to slow, memory-hungry PyTorch ops.

Best Practices

Sampling: temperature=0.6, top_p=0.95, top_k=20. Greedy decoding on long generations is a known repetition-loop failure mode for reasoning models in this class.
Output length: allow generous max_new_tokens (16,384 recommended); every answer opens with a <think> block. Parse and strip the <think>...</think> span for end users.
Scope: the trace mix emphasizes mathematics, reasoning, and instruction following; for the strongest code performance in the family, use Qwen3.8-9B. The fine-tune is text-only; vision behavior is inherited from the base and was not evaluated here.

Stay in the loop

Support / Donate

If this model helped you, consider supporting the project:

BTC: bc1qx6zepu6sfkvshgdmc4ewu6pk6rpadvpgffpp7v
LTC: ltc1qv2mefzps2vtjcpwfx8xxdrpplrcvltswm68r7x

Provenance & licensing

Weights are released under Apache-2.0, inherited from the Qwen3.5-4B base. Shared for research and experimentation, as-is.

Acknowledgements

Developed and released by Empero
Base model: Qwen3.5-4B (Alibaba Qwen team)
Training: TRL + Transformers
Linear-attention kernels: flash-linear-attention, causal_conv1d
Evaluation: lm-evaluation-harness (EleutherAI)

Downloads last month: 122

Safetensors

Model size

5B params

Tensor type

BF16

Model tree for empero-ai/Qwen3.8-4B

Base model

Qwen/Qwen3.5-4B-Base

Finetuned

Qwen/Qwen3.5-4B

Finetuned

(484)

this model

Quantizations

1 model