atrost/nanochat-d12-nested-kl-long-20260413

This repo contains the full nested Nanochat d12 checkpoint at the repo root and extracted prefix submodels in subfolders.

Loading with Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "atrost/nanochat-d12-nested-kl-long-20260413"
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo_id, trust_remote_code=True, torch_dtype="auto")

small = AutoModelForCausalLM.from_pretrained(
    repo_id,
    subfolder="prefix-1",
    trust_remote_code=True,
    torch_dtype="auto",
)

Available extracted prefixes: prefix-1, prefix-2, prefix-3, prefix-4, prefix-5.

trust_remote_code=True is required because this model uses a custom nested architecture and tokenizer.