Instructions to use justinj92/gpt-oss-nemo-20b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use justinj92/gpt-oss-nemo-20b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="justinj92/gpt-oss-nemo-20b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("justinj92/gpt-oss-nemo-20b")
model = AutoModelForCausalLM.from_pretrained("justinj92/gpt-oss-nemo-20b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use justinj92/gpt-oss-nemo-20b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "justinj92/gpt-oss-nemo-20b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "justinj92/gpt-oss-nemo-20b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/justinj92/gpt-oss-nemo-20b

SGLang

How to use justinj92/gpt-oss-nemo-20b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "justinj92/gpt-oss-nemo-20b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "justinj92/gpt-oss-nemo-20b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "justinj92/gpt-oss-nemo-20b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "justinj92/gpt-oss-nemo-20b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use justinj92/gpt-oss-nemo-20b with Docker Model Runner:
```
docker model run hf.co/justinj92/gpt-oss-nemo-20b
```

Question about fine-tuning

by randomanon000 - opened Sep 16, 2025

Discussion

randomanon000

Sep 16, 2025

•

edited Sep 16, 2025

Greetings,

I'm looking to fine-tune the GPT OSS 20B to be fluent in Serbian.
Since I'm a beginner, I was wondering, how long did it take to fine-tune the model on 4xH100 with the dataset of this size and can you share your training configuration by any chance?
Thanks.

justinj92

Owner Sep 16, 2025

It took about 12 hours or so but had few crashes here and there so the overall time it took for me in compute hours was about 28 or so.

Learning Rate: 2e-4
Batch Size: 4 per device (16 total with 4 GPUs)
Gradient Accumulation: 4 steps
Epochs: 4
Max Sequence Length: 2048 tokens
Warmup Ratio: 3%
LR Scheduler: Cosine with minimum LR (10% of peak)
Gradient Checkpointing: Enabled

my suggestion would be to use https://github.com/axolotl-ai-cloud/axolotl or https://github.com/hiyouga/LLaMA-Factory - helps you speed up your training process if you are getting started as beginner.

randomanon000

Sep 19, 2025

Thank you so much!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment