Hugging Face's logo

amd
/
Muse-Spark-Open-816B-A42B-Quark-MXFP4

Safetensors
muse_spark
8-bit precision
quark
Model card Files
xet
 Community
1

Model Overview

  • Model Architecture: MuseSparkForConditionalGeneration
    • Input: Text, Image, Video, Audio
    • Output: Text
  • Supported Hardware Microarchitecture: AMD MI350/MI355
  • Operating System(s): Linux
  • Inference Engine: SGLang / vLLM
  • Model Optimizer: AMD-Quark(v0.12)
    • Quantized layers: MoE experts, except layer 0 and layer 61.
    • Weight quantization:
      • MOE OCP MXFP4, Static
    • Activation quantization:
      • MOE OCP MXFP4, Dynamic

This checkpoint quantizes the routed MoE experts to OCP MXFP4 (except those in layers 0 and 61) and keeps everything else in bf16.

Model Quantization

The model was quantized from meta-models/Muse-Spark-Open-816B-A42B using AMD-Quark. The weights and activations are quantized to MXFP4.

Quantization scripts:

from quark.torch import ModelQuantizer
from quark.torch.quantization.config.template import LLMTemplate
from quark.torch.quantization.weight_convert import SplitFusedExperts, WeightConverter

LLMTemplate.register_template(
    LLMTemplate(
        model_type="muse_spark",
        kv_layers_name=["*k_proj", "*v_proj"],
        q_layer_name="*q_proj",
        exclude_layers_name=[
            "lm_head",
            "*self_attn*",
            "*mlp.gate",
            "*pre_expert_proj",
            "*post_expert_proj",
            "*vision*",
            "*audio*",
            "*language_model.layers.0.*",
            "*language_model.layers.61.*",
        ],
        f2f_weight_converters=[
            WeightConverter(
                "gate_up_proj",
                ["gate_proj.weight", "up_proj.weight"],
                operations=[SplitFusedExperts(split_axis=0)],
            ),
            WeightConverter(
                "down_proj",
                ["down_proj.weight"],
                operations=[SplitFusedExperts(split_axis=0)],
            ),
        ],
    )
)

template = LLMTemplate.get("muse_spark")
quant_config = template.get_config(scheme="mxfp4")

quantizer = ModelQuantizer(quant_config)
quantizer.direct_quantize_checkpoint(
    pretrained_model_path="meta-models/Muse-Spark-Open-816B-A42B",
    save_path="amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4",
    weight_converters=template.f2f_weight_converters,
)

Deployment

The following ROCm recipes serve this checkpoint on AMD Instinct MI355X GPUs. The SGLang recipe is tuned for low latency with DFlash V2 speculative decoding; the vLLM recipe uses tensor parallelism and serves the language model path. For the latest flags and alternative strategies, see the SGLang cookbook and vLLM recipe.

SGLang: MI355X low-latency recipe

The selected recipe uses TP4 + EP4, AITER attention, FlyDSL MoE, and the RadixArk/Muse-Spark-Open-DFlash2 draft model.

docker pull lmsysorg/sglang-rocm:muse-spark-open

docker run \
  --device=/dev/kfd --device=/dev/dri \
  --group-add video \
  --cap-add=SYS_PTRACE --security-opt seccomp=unconfined \
  --shm-size 32g \
  -p 30000:30000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --env SGLANG_USE_AITER=1 \
  --env SGLANG_USE_AITER_UNIFIED_ATTN=1 \
  --env MUSE_MOE_BACKEND=flydsl \
  --ipc=host \
  lmsysorg/sglang-rocm:muse-spark-open \
  sglang serve \
  --model-path amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4 \
  --tp-size 4 \
  --ep-size 4 \
  --attention-backend aiter \
  --page-size 16 \
  --mem-fraction-static 0.85 \
  --host 0.0.0.0 \
  --port 30000 \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path RadixArk/Muse-Spark-Open-DFlash2 \
  --speculative-draft-attention-backend triton \
  --speculative-num-draft-tokens 8 \
  --reasoning-parser muse \
  --tool-call-parser muse

vLLM: MI355X MXFP4 recipe

The selected recipe uses TP8, AITER kernels, chunked prefill, and a 32,768-token serving limit. --language-model-only configures the ROCm path for text serving.

docker pull vllm/vllm-openai-rocm:muse-spark-open

docker run --device=/dev/kfd --device=/dev/dri \
  --security-opt seccomp=unconfined --group-add video \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e VLLM_ROCM_USE_AITER=1 \
  -e VLLM_ROCM_MUSE_MOE_REDUCE_DTYPE=bf16 \
  -e VLLM_USE_BREAKABLE_CUDAGRAPH=1 \
  -e VLLM_USE_RUST_FRONTEND=1 \
  vllm/vllm-openai-rocm:muse-spark-open \
  amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4 \
  --tensor-parallel-size 8 \
  --dtype bfloat16 \
  --distributed-executor-backend mp \
  --language-model-only \
  --compilation-config '{"cudagraph_mode":"FULL_AND_PIECEWISE"}' \
  --no-enable-prefix-caching \
  --enable-chunked-prefill \
  --attention-backend ROCM_AITER_FA \
  --max-model-len 32768 \
  --max-num-seqs 128 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.90 \
  --reasoning-parser muse \
  --enable-auto-tool-choice \
  --tool-call-parser muse

Send a request

Use port 30000 for the SGLang command or port 8000 for the vLLM command.

curl http://localhost:30000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4",
    "messages": [{"role": "user", "content": "Write a short poem about AMD Instinct GPUs."}],
    "reasoning_effort": "high"
  }'

If authentication is required in your environment, pass HF_TOKEN to the container through an environment variable rather than embedding the token in the command.

For additional background, see the SGLang AMD support announcement.

Evaluation

The model was evaluated with SGLang and EvalScope on GPQA Diamond, AIME26, MMMU Pro, SciCode, AA-LCR and IFBench. The base model was evaluated with the identical framework, settings and host.

Accuracy

Benchmark meta-models/Muse-Spark-Open-816B-A42B amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4 (this model) Recovery
GPQA Diamond (mean@8) 90.66 91.04 100.4%
AIME26 (mean@8) 96.67 97.50 100.9%
MMMU Pro, standard (10 options) (mean@2) 80.99 81.10 100.1%
MMMU Pro, vision (mean@2) 81.02 81.74 100.9%
SciCode, subproblem pass rate (mean@8) 51.03 50.69 99.3%
AA-LCR (mean@8) 82.75 82.87 100.1%
IFBench, prompt_level_strict (mean@2) 74.67 75.67 101.3%

SciCode is reported as the subproblem pass rate; the stricter main problem pass rate is 18.27 for the base model and 18.65 for this model. All runs use stochastic sampling, so scores vary from run to run.

Reproduction

Settings

The inference parameters recommended for Muse Spark Open, applied to every run: temperature=1.0, top_p=1.0, top_k=-1 (disabled), reasoning_effort=xhigh.

  • top_k must be sent as -1 explicitly. SGLang's default --sampling-defaults model reads generation_config.json from the checkpoint and applies top_k: 64. Passing --sampling-defaults openai to the server has the same effect without per-request overrides.
  • reasoning_effort must be sent explicitly. The chat template falls back to medium when the field is absent.
  • Context length is 163840, so prompt + max_tokens must fit inside it or the server returns HTTP 400. max_tokens is tuned per benchmark below for this reason.

Environment

  • GPUs: 4x MI355X (this model, TP4/EP4), 8x MI355X (base model, TP8/EP8)
  • Container image: lmsysorg/sglang-rocm:muse-spark-open@sha256:72065dfc0b87818ab415f96c0fc08848233d25f22b002ac8f28c33fad6ec82a4
  • ROCm 7.2.0, PyTorch 2.9.1+rocm7.2.0, AITER 4ad998328
  • SGLang 0.5.21+gc722c0a1
  • AMD-Quark 0.12.post1+f7d8cef
  • EvalScope 1.14.1

Launching server

The SGLang recipe from the Deployment section above, with three additions: --served-model-name amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4, --chat-template <model>/chat_template.jinja and --context-length 163840.

The base model was served with the same recipe, differing only in the model path, --tp-size 8 --ep-size 8 and --mem-fraction-static 0.95.

Evaluating model in a new terminal

GPQA Diamond:

evalscope eval \
  --model amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4 \
  --api-url http://localhost:30000/v1/chat/completions \
  --api-key EMPTY \
  --datasets gpqa_diamond \
  --repeats 8 \
  --eval-batch-size 32 \
  --generation-config '{"temperature":1.0,"top_p":1.0,"top_k":-1,"reasoning_effort":"xhigh","max_tokens":131072,"timeout":7200}' \
  --work-dir logs/gpqa_diamond

AIME26:

evalscope eval \
  --model amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4 \
  --api-url http://localhost:30000/v1/chat/completions \
  --api-key EMPTY \
  --datasets aime26 \
  --repeats 8 \
  --eval-batch-size 48 \
  --generation-config '{"temperature":1.0,"top_p":1.0,"top_k":-1,"reasoning_effort":"xhigh","max_tokens":98304,"timeout":7200}' \
  --work-dir logs/aime26

MMMU Pro — run twice; the reported score is the mean of the two runs:

# standard (10 options); for vision use "dataset_format":"vision"
evalscope eval \
  --model amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4 \
  --api-url http://localhost:30000/v1/chat/completions \
  --api-key EMPTY \
  --datasets mmmu_pro \
  --dataset-args '{"mmmu_pro":{"extra_params":{"dataset_format":"standard (10 options)"}}}' \
  --eval-batch-size 48 \
  --generation-config '{"temperature":1.0,"top_p":1.0,"top_k":-1,"reasoning_effort":"xhigh","max_tokens":98304,"timeout":7200}' \
  --work-dir logs/mmmu_pro_standard

SciCode — executes generated code, so the client needs access to a Docker socket:

evalscope eval \
  --model amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4 \
  --api-url http://localhost:30000/v1/chat/completions \
  --api-key EMPTY \
  --datasets scicode \
  --repeats 8 \
  --eval-batch-size 48 \
  --generation-config '{"temperature":1.0,"top_p":1.0,"top_k":-1,"reasoning_effort":"xhigh","max_tokens":20000,"timeout":7200}' \
  --dataset-args '{"scicode":{"review_timeout":1800,"extra_params":{"provide_background":true}}}' \
  --sandbox '{"enabled":true,"engine":"docker","pool_size":4}' \
  --work-dir logs/scicode

AA-LCR — judged by a second model; any OpenAI-compatible endpoint works. max_tokens must be 32768 or lower, because AA-LCR prompts run from 72K to 115K tokens and a larger completion cap pushes every request past the context limit.

judge.json:

{
  "strategy": "llm",
  "models": [
    {
      "model_id": "<JUDGE_MODEL_ID>",
      "api_url": "<JUDGE_OPENAI_COMPATIBLE_ENDPOINT>",
      "api_key": "<JUDGE_API_KEY>",
      "eval_type": "openai_api",
      "generation_config": {
        "reasoning_effort": "medium",
        "max_tokens": 4096,
        "timeout": 7200
      }
    }
  ]
}

evalscope eval \
  --model amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4 \
  --api-url http://localhost:30000/v1/chat/completions \
  --api-key EMPTY \
  --datasets aa_lcr \
  --repeats 8 \
  --eval-batch-size 8 \
  --generation-config '{"temperature":1.0,"top_p":1.0,"top_k":-1,"reasoning_effort":"xhigh","max_tokens":32768,"timeout":7200}' \
  --dataset-hub huggingface \
  --dataset-args '{"aa_lcr":{"hf_dataset_revision":"bdae010bbce259820c0e34c1d7cce210d966fb75"}}' \
  --judge "$(cat judge.json)" \
  --work-dir logs/aa_lcr

EvalScope does not mask judge credentials: they are written in cleartext to <work-dir>/<timestamp>/configs/task_config.yaml and to the run log, and are visible in ps while the run is in flight.

IFBench — scoring is rule-based and needs emoji, syllapy and nltk in the client environment. Run twice; the reported score is the mean of the two runs.

evalscope eval \
  --model amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4 \
  --api-url http://localhost:30000/v1/chat/completions \
  --api-key EMPTY \
  --datasets ifbench \
  --eval-batch-size 48 \
  --generation-config '{"temperature":1.0,"top_p":1.0,"top_k":-1,"reasoning_effort":"xhigh","max_tokens":131072,"timeout":7200}' \
  --work-dir logs/ifbench

License

Modifications Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.

Built with Muse Spark Open.

Downloads last month
7,719
Safetensors
Model size
435B params
Tensor type
F32
·
BF16
·
U8
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4

Base model

meta-models/Muse-Spark-Open-816B-A42B
Quantized
(25)
this model
Quantizations
1 model