Instructions to use lianghsun/Llama-3.2-Taiwan-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lianghsun/Llama-3.2-Taiwan-3B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lianghsun/Llama-3.2-Taiwan-3B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("lianghsun/Llama-3.2-Taiwan-3B")
model = AutoModelForCausalLM.from_pretrained("lianghsun/Llama-3.2-Taiwan-3B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use lianghsun/Llama-3.2-Taiwan-3B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lianghsun/Llama-3.2-Taiwan-3B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/Llama-3.2-Taiwan-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/lianghsun/Llama-3.2-Taiwan-3B

SGLang

How to use lianghsun/Llama-3.2-Taiwan-3B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lianghsun/Llama-3.2-Taiwan-3B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/Llama-3.2-Taiwan-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lianghsun/Llama-3.2-Taiwan-3B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/Llama-3.2-Taiwan-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use lianghsun/Llama-3.2-Taiwan-3B with Docker Model Runner:
```
docker model run hf.co/lianghsun/Llama-3.2-Taiwan-3B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Model Card for Llama-3.2-Taiwan-3B

Llama-3.2-Taiwan-3B 是以 meta-llama/Llama-3.2-3B 為基底，針對繁體中文與中華民國台灣語境完成持續預訓練（continued pretraining, CPT）之中型基底模型。它是 Llama-3.2-Taiwan-3B-Instruct、Llama-3.2-Taiwan-Legal-3B-Instruct 等下游模型的繁中底座。

⚠️ 規格重點：本模型為 3B 參數、純文本單模態，僅做 CPT、未做指令微調，需自行 SFT 後才有對話能力。

Model Details

Llama-3.2 3B 是兼顧能力與部署成本的良好基底，但其原生繁中能力對台灣本地語境（用詞、文化、法律、社會制度）覆蓋仍不足。Llama-3.2-Taiwan-3B 將大量繁中與台灣本地語料注入模型，作為下游各類繁中應用（一般對話、法律、教育、金融）的共同底座，使 SFT 階段可以更專注在任務與風格的對齊。

核心特點 (Key Features)

繁中底座：訓練語料涵蓋繁中網頁、教育、生活、政府公開文本等多領域內容。
可衍生家族模型：作為一般指令版（Instruct）、法律專業版（Legal）、量化版（GGUF）的共同基底。
3B 規模：在能力與部署成本之間取得平衡。

Model Description

Developed by: Liang Hsun Huang
Funded by: APMIC
Base model: meta-llama/Llama-3.2-3B
Model type: LlamaForCausalLM (Transformers)
Language(s) (NLP): Traditional Chinese, English
License: llama3.2

Model Sources

Repository: lianghsun/Llama-3.2-Taiwan-3B

Citation

@misc{llama_3_2_taiwan_3b,
  title        = {Llama-3.2-Taiwan-3B: A Traditional Chinese Continued-Pretrained Llama-3.2 3B Model for Taiwan},
  author       = {Huang, Liang Hsun},
  year         = {2024},
  howpublished = {\url{https://huggingface.co/lianghsun/Llama-3.2-Taiwan-3B}}
}