GPT-OSS 120B Tokenizer
Standalone tokenizer extracted from openai/gpt-oss-120b.
Useful for tokenization without loading the full model.
Usage
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("srinivas-652/gpt-oss-120b-tokenizer")
tokens = tokenizer("Hello, world!")
