Sage-A1.5-NVFP4

Mixed precision: NVFP4 (W4A4) on every routed expert, FP8 on all other language-model linear layers, BF16 for the rest. 24.21 GB on disk, against 71.93 GB for the BF16 original.

This is a quantized build of InternScience/Sage-A1.5. Only the numeric precision changed: the architecture, tokenizer, chat template and MTP layer are the original ones. Lower precision still changes some outputs; the paired comparison with BF16 is below.

Measured against the BF16 original (paired, same server, held-out data):

• Generated tokens track BF16 closely: 96–99% top-1 agreement on reasoning, tool calls and answers.
• There is no detectable loss on agentic next-action or IFEval.
• BFCL tool-call accuracy is 2.1 points lower (95% CI −3.9 to −0.3).
• Decode is about 1.9× faster than BF16.

GSM8K and MMLU-Pro are not compared yet. The InternScience benchmark chart below belongs to the original BF16 model.

About Sage-A1.5

Sage-A1.5 is a 35B mixture-of-experts agentic model with about 3B parameters active per token. It comes from InternScience at Shanghai AI Laboratory and builds on Qwen3.6-35B-A3B and the earlier Agents-A1. It targets long-horizon work: deep search, software engineering, scientific research, instruction following and tool use. It has a 256K-token context, which the original card extends to 512K with YaRN.

Three capabilities set it apart from Agents-A1:

• Agentic orchestration. The model can split a task into independent sub-questions and hand them to sub-agents that run in parallel, each with its own context and tools. It then merges what they return.
• Agentic verification. An opt-in mode in which the model stops partway through its own reasoning, reviews what it has established and what has failed, and changes course in the same turn, with no external critic.
• Open scientific discovery. It goes beyond answering fixed questions to exploratory research that produces analyses and research artifacts others can inspect.

According to the project page, training runs in three stages over a domain-grounded graph of knowledge and actions built from agent traces:

1. supervised fine-tuning across all domains;
2. domain-level teacher models;
3. on-policy distillation from several teachers at once.

The project page also hosts interactive showcases, from agentic orchestration and verification to end-to-end research in neuroscience, chemistry and physics.

Links: original model card · project page · technical report · GitHub (code and evaluation framework) · Hugging Face collection

Reported results (original BF16 model)

Chart from InternScience's Sage-A1.5 repository. It compares Sage-A1.5 with A1 and Ornith-1.5 (35B class) and with Kimi-K3, GLM-5.3 and GPT-5.6-sol (larger models) on twelve engineering, scientific, agentic, search and instruction-following benchmarks. These scores were measured on the BF16 weights, not on this NVFP4 build.

The original card reports further BF16 results: GAIA 95.1, XBench-DS-2510 92.0, AA-LCR 81.0 and OmniDocBench 91.5. It has the full table and the evaluation settings.

Evaluation against BF16

Run on 2026-10-25 on one NVIDIA DGX Spark (GB10). The BF16 original and this build were served one after the other by the same vLLM build (0.31.1rc1.dev88+g81198e97), with the same flags, prompts and chat template:

• --max-model-len 40960 --max-num-batched-tokens 4096 --gpu-memory-utilization 0.8, MTP off.
• KV cache: BF16 for the original; FP8 for this build, plus one extra pass with a BF16 KV cache.
• Held-out data only: none of the items is in the calibration set. The calibration draw was replayed to check this.

Task accuracy (paired)

Task	Items	Thinking	Max tokens	BF16	NVFP4	Difference, pp (95% CI)	McNemar p
Agentic next action	460	on	4,096	50.0	52.2	+2.2 (−2.4 to +6.7)	0.40
BFCL v3 AST	1,000	off	1,024	83.5	81.4	−2.1 (−3.9 to −0.3)	0.03
IFEval, prompt-level strict	541	off	2,048	83.2	82.8	−0.4 (−3.0 to +2.2)	0.89
IFEval, prompt-level loose	541	off	2,048	87.4	87.1	−0.4
GSM8K	500			not run yet
MMLU-Pro	1,008			not run yet

• Agentic next action: 460 decision points from 93 held-out Nemotron-SFT-Agentic-v2 web-search traces. Each model writes the next assistant turn from the true history. It is correct when its web-search query has word-level F1 ≥ 0.5 with the query in the trace (web-search is the only tool offered). The two builds write the identical query 28% of the time (mean query F1 between them 0.64). Their mean F1 against the trace is about the same: 0.46 for BF16, 0.48 for NVFP4.
• BFCL v3 AST: the simple, multiple, parallel and parallel_multiple categories (400 / 200 / 200 / 200 items), scored with BFCL's own AST checker.
• IFEval: all 541 prompts, scored with lm-eval's IFEval scorer.
• Method:
  • greedy decoding (temperature 0) with Sage's presence penalty of 1.5;
  • a truncated answer counts as wrong; truncation was under 1% for both builds on every task;
  • the confidence interval is a paired bootstrap (10,000 resamples), and the p-value an exact McNemar test on the discordant items.

Output fidelity (teacher-forced)

Both models scored the same held-out text, token by token, through vLLM's prompt_logprobs (top 100):

• Perplexity ratio: NVFP4 over BF16 on the true next token. 1.00 means equal; below 1, this build was more confident than BF16.
• Top-1 agreement: both models rank the same token first.
• KLD: computed over the union of both models' top-100 tokens, so it is a lower bound.

Held-out text	Tokens	Perplexity ratio	Top-1 agreement	KLD mean	KLD p99
Chat: 200 Nemotron-SFT-Instruction-Following-Chat-v2 conversations	255,393	1.004	94.3%	0.035	0.30
General text: wikitext-2 test	296,815	1.010	92.1%	0.031	0.29
Agentic traces, first 4K tokens (93 traces), all tokens	380,835	0.987	84.4%	0.278	6.04
· reasoning	87,179	1.001	96.2%	0.010	0.10
· tool calls	11,301	0.933	96.7%	0.040	0.92
· prompt (system, tools, user)	56,820	0.961	89.1%	0.104	1.10
· tool output (web pages the model reads)	225,368	0.991	78.0%	0.437	7.18
Same, this build with a BF16 KV cache	380,835	0.981	84.3%	0.277	6.03
Long agentic traces, full length up to 32K (16 traces), all tokens	522,293	0.960	79.3%	0.343	5.74
· reasoning	66,429	1.002	96.3%	0.008	0.08
· tool calls	8,212	0.981	98.7%	0.010	0.19
· answers	1,260	1.005	97.9%	0.006	0.07
· tool output	434,702	0.955	76.0%	0.408	6.06
· positions 0–4K	65,520	0.977	83.0%	0.239	4.83
· positions 4K–16K	196,608	0.964	78.5%	0.358	5.86
· positions 16K–32K	260,165	0.953	79.0%	0.357	5.85

Reading the fidelity results:

• On the tokens the model writes (reasoning, tool calls, answers), this build tracks BF16 closely at every length tested: top-1 agreement 96–99%, mean KLD 0.04 or less.
• The larger divergence is in tool output: web text the model reads but never writes, where both models are uncertain. There this build's perplexity is slightly lower than BF16's, not higher.
• The gap does not grow past 4K tokens.
• The FP8 KV cache adds nothing measurable: mean KLD 0.278 with it, 0.277 with a BF16 KV cache.

Speed and size

Measured on the evaluation server above, MTP off.

	BF16	NVFP4	Ratio
Decode, 1 stream, 512 tokens (tokens/s)	28.3	53.0	1.87×
Decode, 4 streams, aggregate (tokens/s)	99.6	158.9	1.60×
Prefill, 8K-token prompt (tokens/s)	4,105	6,123	1.49×
Time to first token, 1 stream (s)	0.146	0.098	0.67×
Checkpoint on disk (GB)	71.93	24.21	0.34×

With MTP on (3 draft tokens), one stream decodes at about 54 tokens/s; see Measured on a DGX Spark.

What is quantized

Part	Format	Size
Routed experts, all 40 layers (256 experts each)	NVFP4: E2M1 weights, one E4M3 scale per 16 values plus one FP32 scale per tensor; activations quantized to NVFP4 dynamically (W4A4)	18.12 GB
Linear attention in_proj_qkv, in_proj_z, out_proj (90 projections)	FP8 E4M3, weights and activations, per-tensor static scales	1.02 GB
Full attention q / k / v / o (40 projections)	FP8 E4M3	0.27 GB
Shared experts (120 projections)	FP8 E4M3	0.13 GB
KV cache (10 full-attention layers)	FP8 E4M3, calibrated scales	n/a
Embeddings, lm_head	BF16	1.02 + 1.02 GB
Routers, shared-expert gates, linear attention in_proj_a / in_proj_b / conv1d, norms	BF16	0.04 GB
Vision tower	BF16, unchanged	0.89 GB
MTP layer (785 tensors)	BF16, byte-identical to the original	1.69 GB
Total		24.21 GB (21.61 GB of text weights without vision and MTP)

How it was made

• Tool: NVIDIA Model Optimizer 0.47.0, with its built-in recipe huggingface/qwen3_5_moe/ptq/nvfp4_experts_mse-fp8_rest-kv_fp8. The expert weights get static NVFP4 scales from an MSE search over FP8 scale values; activation and FP8 scales are max-calibrated. transformers 5.17.0.

• Source: quantized once, directly from the original BF16 weights at revision 0e3205cf441cf01df0c0d238cc4738d0e43b4ca7. The download was checked file by file against the Hub's sha256.

• Calibration: 512 samples of up to 2,048 tokens, batch size 1 (no padding):

  • 320 agentic web-search trajectories from nvidia/Nemotron-SFT-Agentic-v2 (search split), with their tool definitions, tool calls and reasoning;
  • 192 instruction-following chats from nvidia/Nemotron-SFT-Instruction-Following-Chat-v2 (reasoning_off).

  All of them were rendered with Sage-A1.5's own chat template. The agentic share matches what the model is for. Calibration data is not redistributed here.

• Checks: a layout audit of the exported checkpoint (every tensor's format matches the table above), the MTP tensors compared byte for byte with the original, and the sha256 of every file in SHA256SUMS.

Running it with vLLM

Tested only with a vLLM nightly build (0.31.1rc1.dev88+g81198e97) on an NVIDIA DGX Spark (GB10, sm_121). The checkpoint uses ModelOpt's MIXED_PRECISION format, which vLLM loads with --quantization modelopt.

vllm serve kvallespin/Sage-A1.5-NVFP4 \
  --quantization modelopt \
  --kv-cache-dtype fp8 \
  --max-model-len 65536 \
  --gpu-memory-utilization 0.5 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 3}'

The measured run below also used --compilation-config '{"mode": 0, "cudagraph_mode": "FULL_DECODE_ONLY"}' --max-num-seqs 4. Raise --max-model-len up to 262144 for the full native context; the FP8 KV cache needs about 4.2 GiB per 256K tokens. Add --language-model-only to skip the vision tower and save about 0.9 GB.

Sage-A1.5's modes and settings with this build

The chat template, tokenizer and MTP layer are the original ones, so the model's options work as described in the original card. They were not separately evaluated on this build.

• Thinking: on by default. Turn it off per request with "chat_template_kwargs": {"enable_thinking": false}.
• Verification mode: opt-in per request with "chat_template_kwargs": {"verification": true} (a JSON boolean). It needs a non-empty system message. The self-review appears in reasoning_content.
• Agentic orchestration: turned on by giving the model a spawn_subagent tool. Your own harness runs the sub-tasks in parallel and returns their structured results. The original card has the tool schema and the harness settings.
• Sampling (from the original card): temperature 1.0, top_p 0.95, top_k 20, min_p 0.0, presence_penalty 1.5, repetition_penalty 1.0.

Measured on a DGX Spark (GB10)

One server, --gpu-memory-utilization 0.5, FP8 KV cache, MTP with 3 draft tokens.

Weights in GPU memory (vision and MTP included)	22.67 GiB
Weights plus non-torch memory	25.27 GiB
Peak activations, plus CUDA graphs	3.31 GiB + 0.21 GiB
Fixed footprint before KV cache	about 28.8 GiB
KV cache	about 16.6 KiB per token (32.27 GiB held 2,043,013 tokens)
Cold start to a healthy server	270 s
Decode, one request, 512 tokens, thinking off (prefill included)	54.1 tokens/s
MTP draft acceptance	58.9% overall; 79.6% / 57.5% / 39.4% by draft position

Reproduction note

transformers 5.17 opens every checkpoint shard through safetensors' mmap backend and keeps the handles open until loading ends. With safetensors 0.8.0 that backend kept anonymous host memory for everything read until the handles closed. Loading the 72 GB BF16 model onto a unified-memory GPU therefore needed a second full copy in RAM. Switching those safe_open calls to backend="pread" removed it: 0.09 GB of host memory against 2.21 GB with mmap, measured on a 2.2 GB test checkpoint.

Limitations

• BFCL tool-call accuracy is 2.1 points below BF16 (95% CI −3.9 to −0.3). If exact function-call formatting matters most to you, test on your own tools.
• Not yet compared with BF16:
  • GSM8K (math) and MMLU-Pro (knowledge);
  • long-horizon agent benchmarks (for example SWE-bench, GAIA, BrowseComp);
  • the verification and orchestration modes.
• All comparisons used greedy decoding. Sage's recommended sampling (temperature 1.0) was not evaluated.
• Tested only on GB10 (sm_121) with the vLLM build named above. Other GPUs and vLLM versions are untested.
• The vision tower is unchanged, but image understanding was not evaluated here.

Credits and license

• InternScience for Sage-A1.5, and the Qwen team for Qwen3.6-35B-A3B.
• NVIDIA for Model Optimizer and the Nemotron calibration datasets.
• The vLLM project.

Licensed Apache 2.0, the same license as InternScience/Sage-A1.5. Changes from the original: the weights were quantized as described above, and config.json gained the matching quantization_config (with hf_quant_config.json alongside). The tokenizer, chat template, processor configs and MTP layer are unchanged.

If you use this model, please cite the original work:

@misc{lou2026sagea15longhorizonexecutionscientific,
      title={Sage-A1.5: From Long-Horizon Execution to Scientific Exploration with Capability Composition},
      author={Sage-A1.5 Team},
      year={2026},
      eprint={2610.28578},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2610.28578},
}