Instructions to use FermionResearch/Neutrino-0.6B-Chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FermionResearch/Neutrino-0.6B-Chat with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="FermionResearch/Neutrino-0.6B-Chat")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("FermionResearch/Neutrino-0.6B-Chat", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use FermionResearch/Neutrino-0.6B-Chat with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FermionResearch/Neutrino-0.6B-Chat"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FermionResearch/Neutrino-0.6B-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/FermionResearch/Neutrino-0.6B-Chat

SGLang

How to use FermionResearch/Neutrino-0.6B-Chat with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FermionResearch/Neutrino-0.6B-Chat" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FermionResearch/Neutrino-0.6B-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FermionResearch/Neutrino-0.6B-Chat" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FermionResearch/Neutrino-0.6B-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use FermionResearch/Neutrino-0.6B-Chat with Docker Model Runner:
```
docker model run hf.co/FermionResearch/Neutrino-0.6B-Chat
```

Neutrino-0.6B-Chat / MANIFEST.json

FermionResearch

runners: strict id-parser rebuild (B8) + native --draft everywhere + bin docs: MANIFEST.json

122255e verified 2 months ago

Raw History Blame Contribute Delete

8.57 kB

	{
	"note": "provenance: HF-tree filename -> source + sha256; weights entry is declarative until materialized",
	"weights_container": {
	"file": "neutrino-0.6b-chat_v4.bin",
	"status": "FINAL \u2014 shipping container",
	"bytes": 327719836,
	"sha256": "0ec9810f002c921de324b1a3b6dd124e4f6b56ff7807e6864a4a175036272806",
	"source": "Fermion Research release archive (public copy: huggingface.co/fermionresearch/Neutrino-0.6B-Chat)"
	},
	"files": {
	".gitattributes": {
	"bytes": 1829,
	"sha256": "83f9874d64fc325a932d03f7e71d75895113b7faf57cfe78d759fd8ac4f514c5",
	"source": "Fermion Research release archive"
	},
	"LICENSE": {
	"bytes": 11347,
	"sha256": "2f978a54ef89f3e4ee2d413c347e4b03500c95f29342277a287acc2ae27ae280",
	"source": "Apache-2.0 text (same file across all release trees)"
	},
	"NOTICE": {
	"bytes": 1561,
	"sha256": "6b7f7ed0cbf486ac68240e223d551eabdeb1d6dba59ed614f04316780437e643",
	"source": "Fermion Research release archive"
	},
	"bin/README.md": {
	"bytes": 12057,
	"sha256": "a7f2f338e5d4cf3faae6a9e2e13cb193ed71d019281d9b4cd4c8151e48d6ccb4",
	"source": "2026-07-31 strict-parser rebuild"
	},
	"bin/fermion-run-linux-x64": {
	"bytes": 113296,
	"sha256": "39bc24eafc40f13594f63789b649e3698b326454c63ede118959e6e6197088d7",
	"source": "2026-07-31 strict-parser rebuild"
	},
	"bin/fermion-run-linux-x64.sha256": {
	"bytes": 88,
	"sha256": "dcca4682d2f012df644caee9992ae8b48ab12ec0f1fb1a0ddc84aa865c530158",
	"source": "2026-07-31 strict-parser rebuild"
	},
	"bin/fermion-run-macos-arm64": {
	"bytes": 583400,
	"sha256": "59b59d4df38baa08993a8429829c4889f1234183b1559b0078355f27351a1e66",
	"source": "2026-07-31 strict-parser rebuild"
	},
	"bin/fermion-run-macos-arm64.sha256": {
	"bytes": 90,
	"sha256": "5664fda2394e3c3d1c8ca0a9869e4abe8367c307214a3070bd834457f74626da",
	"source": "2026-07-31 strict-parser rebuild"
	},
	"config.json": {
	"bytes": 508,
	"sha256": "c97949fb6b7e195f0ef58fd16dc0588e47b6afb80efcc0cd7dd0d13be9bbd332",
	"source": "written for the trtc_v4 loader (hf_ternary registration)"
	},
	"generation_config.json": {
	"bytes": 240,
	"sha256": "4e4978dea7d87e5ccb1cc3145420c0ccfa3f129e62cb8dc6cd378811fadfef1d",
	"source": "transformers generation defaults for this SKU"
	},
	"gguf/README.md": {
	"bytes": 1325,
	"sha256": "eaaef843275e3f12895c536c208a788467e2c86ca038e5be84d98bf1cba9239e",
	"source": "Fermion Research release archive"
	},
	"merges.txt": {
	"bytes": 1671853,
	"sha256": "8831e4f1a044471340f7c0a83d7bd71306a5b867e95fd870f74d0c5308a904d5",
	"source": "Qwen/Qwen3-8B tokenizer file (Apache-2.0 upstream)"
	},
	"mlx/README.md": {
	"bytes": 2157,
	"sha256": "49d3a1cc862b316f89e95cfa54f9edb0c3e957c336a6a9d2972f8654bcd20eab",
	"source": "Fermion Research release archive"
	},
	"receipts/c_battery_chatv2_full.json": {
	"bytes": 25198,
	"sha256": "a35e34ecd1b7a531710d9224b7c6f682c97f933b69f6fe10415c6caf1ac281d9",
	"source": "Fermion Research release archive"
	},
	"receipts/eval_c100.json": {
	"bytes": 32861,
	"sha256": "77df0b000066d39e4eb15033ed57eb738c75ca740d60b44945b9772040b74561",
	"source": "Fermion Research release archive"
	},
	"receipts/mlx_battery_chatv2.json": {
	"bytes": 25242,
	"sha256": "0904796cf0175fbd22f7107fee1fe2834366582926f916f60d425ca0d03c5eba",
	"source": "Fermion Research release archive"
	},
	"tokenizer.json": {
	"bytes": 11422654,
	"sha256": "aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4",
	"source": "Qwen/Qwen3-8B tokenizer file (Apache-2.0 upstream)"
	},
	"tokenizer_config.json": {
	"bytes": 9979,
	"sha256": "02e7044c846290e910d9c9deca9812315d64be56a6aeee3cfbedc99d177644c4",
	"source": "Qwen/Qwen3-8B tokenizer file (Apache-2.0 upstream)"
	},
	"vocab.json": {
	"bytes": 2776833,
	"sha256": "ca10d7e9fb3ed18575dd1e277a2579c16d108e32f27439684afa0e10b1440910",
	"source": "Qwen/Qwen3-8B tokenizer file (Apache-2.0 upstream)"
	},
	"mlx/receipts/noflash_fp16_race_20260731.json": {
	"bytes": 1287,
	"sha256": "efd6ad55ee8d994ba509347542ad49c409d78372deb949dcd75f8fe8bac94a67",
	"source": "v10 fermion_mlx runtime (same bytes as Neutrino-0.6B mlx pack, 2026-07-31)"
	},
	"mlx/fermion_mlx/trit_kernels.py": {
	"bytes": 44235,
	"sha256": "ba1c32e18585d8a1e6202c739ba6e4cbcdaefcc23f0dd7d665ffbfcfb7bbd1d4",
	"source": "v10 fermion_mlx runtime (same bytes as Neutrino-0.6B mlx pack, 2026-07-31)"
	},
	"mlx/fermion_mlx/mbatch.py": {
	"bytes": 20068,
	"sha256": "5736d2cf0ef3d7215b92beebc2c4dc3fbdc09f147effd5f7e3b8b696cdfb71a1",
	"source": "v10 fermion_mlx runtime (same bytes as Neutrino-0.6B mlx pack, 2026-07-31)"
	},
	"mlx/fermion_mlx/drafter.py": {
	"bytes": 6892,
	"sha256": "a0c5ace9e4dc94926f46e6495407e9463676d25345884a6eef626dcb92a9de14",
	"source": "v10 fermion_mlx runtime (same bytes as Neutrino-0.6B mlx pack, 2026-07-31)"
	},
	"mlx/fermion_mlx/__init__.py": {
	"bytes": 6626,
	"sha256": "999f003da4e49311413876df3b4011eae515fbf15684a96a4794af8f946d5185",
	"source": "v10 fermion_mlx runtime (same bytes as Neutrino-0.6B mlx pack, 2026-07-31)"
	},
	"mlx/fermion_mlx/spec.py": {
	"bytes": 15875,
	"sha256": "cb92be27e752580f3e9852191d50dc6b421cd5e33d18b7f53b528d4a832c7895",
	"source": "v10 fermion_mlx runtime (same bytes as Neutrino-0.6B mlx pack, 2026-07-31)"
	},
	"mlx/fermion_mlx/gates.py": {
	"bytes": 7411,
	"sha256": "c032e35944e4ec3c0b991e246114697a9538fa9bfcfff183d39b066d3d180349",
	"source": "v10 fermion_mlx runtime (same bytes as Neutrino-0.6B mlx pack, 2026-07-31)"
	},
	"mlx/fermion_mlx/flash_sdpa.py": {
	"bytes": 11367,
	"sha256": "6ce98b11049ee902db14259b3db320f7df32b9be4a1fb13d86a2a76577a30cb1",
	"source": "v10 fermion_mlx runtime (same bytes as Neutrino-0.6B mlx pack, 2026-07-31)"
	},
	"mlx/fermion_mlx/container.py": {
	"bytes": 6059,
	"sha256": "c5b352184ba9ddf9a3e424731226cb7dcb9901d3a1516c67d4557953b0101f2f",
	"source": "v10 fermion_mlx runtime (same bytes as Neutrino-0.6B mlx pack, 2026-07-31)"
	},
	"mlx/fermion_mlx/model.py": {
	"bytes": 26179,
	"sha256": "7d48f63b7043c8e8508a937f0d2e2e4a17a82c02800a082671a7dbae163d0d72",
	"source": "v10 fermion_mlx runtime (same bytes as Neutrino-0.6B mlx pack, 2026-07-31)"
	},
	"mlx/fermion_mlx/arch3.py": {
	"bytes": 8206,
	"sha256": "84a15da8566acdfbf6acef60e09099113db150c0f53deefb04491365ea7efef5",
	"source": "v10 fermion_mlx runtime (same bytes as Neutrino-0.6B mlx pack, 2026-07-31)"
	},
	"mlx/fermion_mlx/MANIFEST.json": {
	"bytes": 1077,
	"sha256": "6f2f11bf17214c7f450f231b377a9b47c446904e86ea670cdce9b328cf472cb8",
	"source": "v10 fermion_mlx runtime (same bytes as Neutrino-0.6B mlx pack, 2026-07-31)"
	},
	"mlx/fermion_mlx/mma.py": {
	"bytes": 14725,
	"sha256": "82905730ea79240d9647e63a8131c076781e72979efe2a05999c911206db8c96",
	"source": "v10 fermion_mlx runtime (same bytes as Neutrino-0.6B mlx pack, 2026-07-31)"
	},
	"mlx/fermion_mlx/__main__.py": {
	"bytes": 13847,
	"sha256": "8ba4104366a29737b5fecc042ecfac88eb0106122464af29c5822b2ee81ba7d9",
	"source": "v10 fermion_mlx runtime (same bytes as Neutrino-0.6B mlx pack, 2026-07-31)"
	},
	"mlx/fermion_mlx/heads.py": {
	"bytes": 9239,
	"sha256": "a9353d3e9bd93130a1533379986b11c7baefa0e11af555c36a94d9f9d0e97226",
	"source": "v10 fermion_mlx runtime (same bytes as Neutrino-0.6B mlx pack, 2026-07-31)"
	},
	"bin/MANIFEST.json": {
	"bytes": 1519,
	"sha256": "259e7bdbc321833096f3eb8a190f73e8753153f9d1a6abb69c785f3dbaffad80",
	"source": "2026-07-31 strict-parser rebuild"
	},
	"mlx/merges.txt": {
	"bytes": 1671853,
	"sha256": "8831e4f1a044471340f7c0a83d7bd71306a5b867e95fd870f74d0c5308a904d5",
	"source": "live artifact, pinned at 2026-07-31 audit"
	},
	"mlx/requirements.txt": {
	"bytes": 191,
	"sha256": "1cf4d1fa581aaa5deae0568f88388c51166bbd69e7a89080b5a53488a7ef39b6",
	"source": "live artifact, pinned at 2026-07-31 audit"
	},
	"mlx/tokenizer.json": {
	"bytes": 11422654,
	"sha256": "aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4",
	"source": "live artifact, pinned at 2026-07-31 audit"
	},
	"mlx/tokenizer_config.json": {
	"bytes": 9979,
	"sha256": "02e7044c846290e910d9c9deca9812315d64be56a6aeee3cfbedc99d177644c4",
	"source": "live artifact, pinned at 2026-07-31 audit"
	},
	"mlx/vocab.json": {
	"bytes": 2776833,
	"sha256": "ca10d7e9fb3ed18575dd1e277a2579c16d108e32f27439684afa0e10b1440910",
	"source": "live artifact, pinned at 2026-07-31 audit"
	}
	}
	}