Instructions to use nkthebass/tinybrainbot-360m-a75m-v5.1-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nkthebass/tinybrainbot-360m-a75m-v5.1-base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nkthebass/tinybrainbot-360m-a75m-v5.1-base")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nkthebass/tinybrainbot-360m-a75m-v5.1-base")
model = AutoModelForCausalLM.from_pretrained("nkthebass/tinybrainbot-360m-a75m-v5.1-base", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use nkthebass/tinybrainbot-360m-a75m-v5.1-base with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf nkthebass/tinybrainbot-360m-a75m-v5.1-base:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf nkthebass/tinybrainbot-360m-a75m-v5.1-base:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf nkthebass/tinybrainbot-360m-a75m-v5.1-base:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf nkthebass/tinybrainbot-360m-a75m-v5.1-base:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf nkthebass/tinybrainbot-360m-a75m-v5.1-base:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf nkthebass/tinybrainbot-360m-a75m-v5.1-base:Q4_K_M

Use Docker

docker model run hf.co/nkthebass/tinybrainbot-360m-a75m-v5.1-base:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use nkthebass/tinybrainbot-360m-a75m-v5.1-base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nkthebass/tinybrainbot-360m-a75m-v5.1-base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nkthebass/tinybrainbot-360m-a75m-v5.1-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/nkthebass/tinybrainbot-360m-a75m-v5.1-base:Q4_K_M

SGLang

How to use nkthebass/tinybrainbot-360m-a75m-v5.1-base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nkthebass/tinybrainbot-360m-a75m-v5.1-base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nkthebass/tinybrainbot-360m-a75m-v5.1-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nkthebass/tinybrainbot-360m-a75m-v5.1-base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nkthebass/tinybrainbot-360m-a75m-v5.1-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Ollama
How to use nkthebass/tinybrainbot-360m-a75m-v5.1-base with Ollama:
```
ollama run hf.co/nkthebass/tinybrainbot-360m-a75m-v5.1-base:Q4_K_M
```
Unsloth Desktop
Docker Model Runner
How to use nkthebass/tinybrainbot-360m-a75m-v5.1-base with Docker Model Runner:
```
docker model run hf.co/nkthebass/tinybrainbot-360m-a75m-v5.1-base:Q4_K_M
```

Lemonade

How to use nkthebass/tinybrainbot-360m-a75m-v5.1-base with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull nkthebass/tinybrainbot-360m-a75m-v5.1-base:Q4_K_M

Run and chat with the model

lemonade run user.tinybrainbot-360m-a75m-v5.1-base-Q4_K_M

List all available models

lemonade list

Atomic Chat
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

TinyBrainBot-360M-A75M-v5.1-Base

A mixture-of-experts base model trained from scratch on 17.9B tokens: 383M parameters in total, 75M active per token (24 experts per layer, 4 used per token). It loads as a standard MixtralForCausalLM (no custom code). It replaces TinyBrainBot-360M-A75M-v4.1-MoE-Base, with a new final training stage built around school-level knowledge, and it beats it on every Open SLM Leaderboard task.

Scores

Zero-shot, full test sets, acc_norm; (raw acc) underneath. Intelligence Index from the Open SLM Leaderboard.

Model	Params	Int Index	HellaSwag	ARC-E	ARC-C	PIQA	ArithMark-3
TinyBrainBot-360M-A75M-v5.1-Base (this model)	383M (75M active)	29.76	38.68	58.96	36.18	68.01	53.00
(raw acc)		27.93	32.75	64.60	33.36	67.68	53.00
TinyBrainBot-360M-A75M-v4.1-MoE-Base (previous)	383M (75M active)	24.40	37.27	55.64	32.34	66.32	42.00
GPT-X3-150M	149M	33.05	46.2	59.6	31.3	71.3	50.9
Boris-2-Preview-2	144M	30.58	46.6	59.1	30.2	70.8	42.2
KLM-1.5	99M	30.26	43.7	59.7	32.7	68.5	48.2
Sol Pro	138M	28.60	46.7	54.0	31.8	71.0	36.0

Other models' numbers are from the leaderboard.

Arithmetic it never trained on

On 1,380 held-out arithmetic problems in topics and wordings the model never saw, it scores 47.3, close to its public ArithMark-3 of 53.0.

Quick start

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("nkthebass/tinybrainbot-360m-a75m-v5.1-base")
model = AutoModelForCausalLM.from_pretrained("nkthebass/tinybrainbot-360m-a75m-v5.1-base")
ids = tok("The water cycle begins when the sun heats water in oceans and lakes, causing it to", return_tensors="pt")
print(tok.decode(model.generate(**ids, max_new_tokens=30, do_sample=False, repetition_penalty=1.0)[0]))

GGUF files for llama.cpp, LM Studio, Ollama and Jan: F16 (768 MB), Q8_0 (409 MB) and Q4_K_M (239 MB). Greedy output matches transformers token for token.

This is a base model: it continues text and does not chat. Its chat version, TinyBrainBot-360M-A75M-v5.1-Instruct, is in the TinyBrainBot v5.1 collection.

Model


Parameters	383.4M total; 75.7M active non-embedding per token + 24.6M tied embedding
Shape	hidden 768, 12 layers, 12 query heads of 64 with 4 KV heads (GQA), RMSNorm, RoPE
Experts	24 SwiGLU experts of width 512 per layer, top-4 routing
Vocabulary	32,000 SentencePiece; line breaks are kept in this version (earlier TinyBrainBot models on this tokenizer never saw them)
Context	2,048 tokens

Training

17.9B tokens: 16.7B of pretraining at constant learning rate, then a new 1.2B-token decay.

Pretraining (16.7B)	Share
Educational web: FineWeb-Edu, DCLM, finepdfs-edu	44%
Wikipedia	13%
Books and stories: Gutenberg, BookCorpus, TinyStories	11%
Math: MegaMath, FineMath, Orca-Math	9%
Code: the-stack-smol	9%
Cosmopedia (Stanford, OpenStax, math)	7%
Q&A, facts and reasoning (our distilled sets)	7%

Decay (1.2B)	Share
Q&A and facts: our distilled sets, GooAQ, FactBase-DCLM, finephrase FAQ, ELI5	26%
School-level textbooks: SmolLM cosmopedia-v2 and Cosmopedia, school audiences	22%
Educational and broad web: top-scored FineWeb-Edu pages, DCLM-Ultra-1.1, ClimbMix, WikiHow	22%
Wikipedia (dense and Simple English)	12%
Grade-school science Q&A	10%
Commonsense, how-to and writing (our distilled sets)	5%
Arithmetic (our drills, Orca-Math, UltraData-Math)	3%

The grade-school science Q&A (about 420k questions with short explained answers, no answer options) was written by Ling-3.0-Flash; any item sharing an 8-word phrase with an ARC question was removed before training. No benchmark train or test data was used.

Recipe: AdamW, peak LR 8e-4 with a warmup-stable-decay schedule, 262k tokens per step, fp16, 2x V100.

Evaluation

lm_eval --model hf --model_args pretrained=nkthebass/tinybrainbot-360m-a75m-v5.1-base,dtype=float32 --tasks hellaswag,arc_easy,arc_challenge,piqa --num_fewshot 0
python bencharithmark-3.py --model nkthebass/tinybrainbot-360m-a75m-v5.1-base --dtype float32

Limitations

A small base model: good at short factual and commonsense continuations for its size, but open-ended generation is limited and can be wrong. Not aligned or safety-tuned.

Downloads last month: 768

Safetensors

Model size

0.4B params

Tensor type

F32

Model tree for nkthebass/tinybrainbot-360m-a75m-v5.1-base

Quantizations

3 models

Datasets used to train nkthebass/tinybrainbot-360m-a75m-v5.1-base

Collection including nkthebass/tinybrainbot-360m-a75m-v5.1-base

TinyBrainBot v5.1

Collection

TinyBrainBot v5.1: 100M and 360M-A75M base + instruct models. The chat models and the 360M base were trained with line breaks. • 7 items • Updated 4 days ago • 1

Evaluation results

acc_norm (0-shot) on ARC-Easy
test set self-reported

58.960
acc_norm (0-shot) on ARC-Challenge
test set self-reported

36.180
acc_norm (0-shot) on HellaSwag
validation set self-reported

38.680
acc_norm (0-shot) on PIQA
validation set self-reported

68.010
acc_norm (0-shot) on ArithMark-3.0
test set self-reported

53.000