Instructions to use ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16
# Run inference directly in the terminal:
./llama-cli -hf ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16

Use Docker

docker model run hf.co/ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16

LM Studio
Jan

vLLM

How to use ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16

Ollama
How to use ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF with Ollama:
```
ollama run hf.co/ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16
```

Unsloth Studio

How to use ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF to start chatting

How to use ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF with Docker Model Runner:
```
docker model run hf.co/ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16
```

Lemonade

How to use ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16

Run and chat with the model

lemonade run user.MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF-BF16

List all available models

lemonade list

Hermes Agent

How to use ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF:BF16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

MiMo-V2.5-Pro DFlash Draft: BF16 GGUF for ik_llama.cpp

A GGUF conversion of the DFlash block-diffusion draft for MiMo-V2.5-Pro, built for speculative decoding in ik_llama.cpp. This is the validation artifact for ik_llama.cpp PR #2048.

Important: This is not a standalone model. It is a 5-layer draft that shares the target's token embedding and output head. It only produces meaningful output when launched as --model-draft alongside a full MiMo-V2.5-Pro target GGUF.

Files

File	Size	Precision	SHA-256
`mimo-v25-pro-dflash-draft-bf16-rope5m-vscale-rope64.gguf`	5,542,740,352 bytes	BF16	`e37bb1afdc5f8cc04df03b6f9d258be32afda44431191de22a056f4ab131a1e9`

Matching target

Pair this draft with a MiMo-V2.5-Pro target GGUF. Validated against:

gghfez/MiMo-V2.5-Pro-ik_llama-unfused-GGUF, file MiMo-V2.5-Pro-IQ3_S-unfused.gguf (IQ3_S, about 349 GiB)

The validation used an unfused target. One fused-MoE conversion tested during the PR produced garbled target-only output, so it was excluded from the DFlash results. That observation does not establish that every fused MiMo conversion is incompatible.

Usage (ik_llama.cpp)

llama-server \
  -m   MiMo-V2.5-Pro-IQ3_S-unfused.gguf \
  --model-draft mimo-v25-pro-dflash-draft-bf16-rope5m-vscale-rope64.gguf \
  -ngl 999 -ngld 999 \
  -sm graph --max-gpu 4 \
  -fa on -ctk q4_0 -ctv q4_0 -ctkd q4_0 -ctvd q4_0 \
  --spec-type dflash:n_max=1,p_min=0.0,cross_ctx=16 \
  -c 8192 --host 127.0.0.1 --port 8080

--spec-type dflash:... selects the DFlash speculator. n_max=1 was optimal in this test. cross_ctx=16 is a small ring buffer of recent target positions visible to the draft; larger values reduced acceptance sharply.
-sm graph --max-gpu 4 was the best multi-GPU profile tested on 4×H200 NVL. Pinning the draft to one GPU with -devd added overhead under graph split.

Conversion metadata

GGUF key	Value	Note
`dflash-draft.block_count`	5	5-layer draft
`dflash-draft.attention.head_count` / `head_count_kv`	128 / 8	GQA
`dflash-draft.rope.dimension_count`	64	`head_dim 128 × partial_rotary_factor 0.5` (upper half is NoPE)
`dflash-draft.rope.freq_base`	10,000	draft Q/K RoPE base from `rope_theta`
`dflash-draft.dflash.backbone_rotary_base`	5,000,000	RoPE base used for captured target-context K
`dflash-draft.attention.value_scale`	0.612
`dflash-draft.dflash.block_size`	8	diffusion block size
`dflash-draft.dflash.mask_token_id`	151669
`dflash-draft.dflash.target_layer_ids`	`[0, 15, 31, 47, 69]`	captured target layers
`dflash-draft.dflash.n_target_features`	30,720	`target hidden_size 6144 × 5 layers`
`dflash-draft.attention.sliding_window`	1024	all-SWA pattern across the 5 layers
I/O contract	`shared-target`	`token_embd` and `output` reuse the target's tensors

The file contains 63 tensors: five 12-tensor draft blocks plus dflash_fc, dflash_hidden_norm, and output_norm. It includes five blk.N.attn_sinks.weight tensors and does not contain token_embd or output tensors.

Validation

Validation was run remotely on 4×H200 NVL for ik_llama.cpp PR #2048. The draft was tested greedily at n_max=1, cross_ctx=16, with q4_0 KV cache on both target and draft.

Prompt	Draft acceptance	DFlash decode (tok/s)	No-spec decode (tok/s)
`double-link-list` fixture	54.6%	55.6	59.9
`quick-sort` fixture	60.4%	59.4	60.8

Conversion notes

The source weights are the BF16 drafter in the dflash/ subdirectory of XiaomiMiMo/MiMo-V2.5-Pro-FP4-DFlash.
Target metadata and the tokenizer are read from the matching MiMo-V2.5-Pro-FP4-DFlash repository root with --target-model-dir. The draft carries no embeddings of its own.
Convert with convert_hf_to_gguf.py from ik_llama.cpp PR #2048, using the DFlashDraftModel path and --outtype bf16.

cd MiMo-V2.5-Pro-FP4-DFlash

python3 ../ik_llama.cpp/convert_hf_to_gguf.py dflash \
  --target-model-dir . \
  --outtype bf16 \
  --outfile mimo-v25-pro-dflash-draft-bf16-rope5m-vscale-rope64.gguf

Use ik_llama.cpp at PR #2048 or later. The final PR fix stores the draft's 10,000 RoPE base separately from the 5,000,000 backbone base. Older converter outputs must be reconverted rather than repaired by changing only rope.freq_base.

Troubleshooting low acceptance

Confirm that the target produces coherent output by itself. A broken target conversion makes DFlash acceptance meaningless.
Run an ik_llama.cpp build that includes PR #2048. The change includes runtime handling for attention sinks, value scaling, and MiMo2 target-feature capture in addition to GGUF metadata support.
Start with dflash:n_max=1,p_min=0.0,cross_ctx=16 and a greedy code-completion prompt comparable to the validation fixtures. The reported 55–60% rates are fixture-specific, not a universal acceptance baseline.
Test the target with this published draft. If this file also gives very low acceptance, inspect the target conversion and runtime setup. If this file works, compare the converted draft against the metadata and 63-tensor contract above.

Downloads last month: 580

GGUF

Model size

3B params

Architecture

dflash-draft

Hardware compatibility

16-bit

Model tree for ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF

Base model

XiaomiMiMo/MiMo-V2.5-Pro

Quantized

(9)

this model