Instructions to use lianghsun/TangYin with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lianghsun/TangYin with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lianghsun/TangYin")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("lianghsun/TangYin")
model = AutoModelForCausalLM.from_pretrained("lianghsun/TangYin", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use lianghsun/TangYin with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lianghsun/TangYin"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/TangYin",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/lianghsun/TangYin

SGLang

How to use lianghsun/TangYin with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lianghsun/TangYin" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/TangYin",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lianghsun/TangYin" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/TangYin",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use lianghsun/TangYin with Docker Model Runner:
```
docker model run hf.co/lianghsun/TangYin
```

TangYin / README.md

lianghsun

docs: align model card with template style

6eff881 verified 6 months ago

preview code

Raw History Blame Contribute Delete

2.97 kB

metadata

license: gemma
language:
  - zh
base_model:
  - lianghsun/gemma-3-tw-270m-it
  - lianghsun/gemma-3-tw-270m
datasets:
  - lianghsun/tw-poetry-chat
library_name: transformers
pipeline_tag: text-generation
tags:
  - Taiwan
  - DevFest
  - poetry
  - zhtw
  - Gemma-3
  - gemma3
  - SLM
  - chat

Model Card for TangYin

TangYin（唐寅） 是以 gemma-3-tw-270m-it 為基底，以中國古典詩詞對話資料（tw-poetry-chat）微調的 270M 級繁體中文詩詞對話模型。模型命名取自明代著名才子畫家「唐寅」，呼應其專長於古典詩詞解析、賞析與創作的設計初衷。

⚠️ 規格重點：本模型為 270M 參數 SLM、純文本單模態，主題聚焦在中國古典詩詞對話。

Model Details

通用模型在古典詩詞領域的表現常流於表面、且常出現張冠李戴的錯誤。TangYin 透過以七言詩等古典詩詞 chat 資料微調，使 270M 級小模型在詩詞翻譯、賞析、典故、用典等任務上具備穩定的表現，適合作為國文教學助理、文學陪聊與在地文化推廣場景的輕量化工具。

本模型是 Google DevFest Taipei 2025 的實作演示之一，用以展示 270M 級小模型透過領域微調可達成的應用效果。

核心特點 (Key Features)

古典詩詞專精：訓練資料聚焦在七言詩等古典詩詞題型，回答結構穩定。
270M 端側可部署：模型小巧，可在筆電 CPU、行動裝置上順暢執行。
DevFest 演示模型：作為「小模型 + 領域微調」設計實踐之公開範例。

Model Description

Developed by: Liang Hsun Huang
Funded by: APMIC
Base model: lianghsun/gemma-3-tw-270m-it
Model type: Gemma3ForCausalLM (Transformers)
Language(s) (NLP): Traditional Chinese（含古文）
License: gemma (Google usage license)
Finetuned from model: lianghsun/gemma-3-tw-270m-it

Model Sources

Repository: lianghsun/TangYin

Citation

@misc{tangyin,
  title        = {TangYin: A Lightweight Classical Chinese Poetry Chat Model},
  author       = {Huang, Liang Hsun},
  year         = {2025},
  howpublished = {\url{https://huggingface.co/lianghsun/TangYin}}
}

Acknowledge

特此感謝 APMIC 的算力支援。

Model Card Authors

Huang Liang Hsun

Model Card Contact

Huang Liang Hsun