Add vLLM, MLX, and llama.cpp deployment docs with working implementations

README.md

@@ -188,13 +188,16 @@ print(response)
 
 ### Pre-quantized Versions
 
-| Version | Repo | VRAM | Format |
-|---------|------|------|--------|
+| Version | Repo | VRAM/Memory | Format |
+|---------|------|-------------|--------|
 | **4-bit NF4** | [`Akahsizrr/fuse-1-Lite-4bit`](https://huggingface.co/Akahsizrr/fuse-1-Lite-4bit) | 3.36 GB | bitsandbytes |
 | **8-bit** | [`Akahsizrr/fuse-1-Lite-8bit`](https://huggingface.co/Akahsizrr/fuse-1-Lite-8bit) | 6.00 GB | bitsandbytes |
 | **bfloat16** | This repo | ~12 GB | safetensors |
+| **MLX** | [`Akahsizrr/fuse-1-Lite-MLX`](https://huggingface.co/Akahsizrr/fuse-1-Lite-MLX) | ~12 GB | MLX safetensors |
+| **GGUF F16** | [`Akahsizrr/fuse-1-Lite-GGUF`](https://huggingface.co/Akahsizrr/fuse-1-Lite-GGUF) | ~11.4 GB | GGUF |
+| **vLLM plugin** | [`Akahsizrr/fuse-1-Lite-vLLM`](https://huggingface.co/Akahsizrr/fuse-1-Lite-vLLM) | ~12 GB | vLLM plugin |
 
-### bitsandbytes 4-bit (NF4) — ~4.5 GB VRAM
+### bitsandbytes 4-bit (NF4) — 3.36 GB VRAM
 
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
@@ -216,7 +219,7 @@ model = AutoModelForCausalLM.from_pretrained(
 tokenizer = AutoTokenizer.from_pretrained("Akahsizrr/fuse-1-Lite")
 ```
 
-### bitsandbytes 8-bit — ~7 GB VRAM
+### bitsandbytes 8-bit — 6.00 GB VRAM
 
 ```python
 from transformers import AutoModelForCausalLM, BitsAndBytesConfig
@@ -242,49 +245,93 @@ model.set_coding_enabled(False)
 model.set_coding_enabled(True)
 ```
 
-### vLLM
+### vLLM — High-Throughput Serving
 
-> **Note:** vLLM does not currently support the custom `Fuse3ForCausalLM` architecture.
-> The model uses LFM2's hybrid conv+attention backbone with augmented MoE layers,
-> which requires a custom vLLM model implementation. Use transformers for inference.
+fuse-1 Lite is supported in vLLM via a plugin that extends vLLM's native LFM2
+implementation with expert augmentation layers.
 
-To use with vLLM, you would need to:
-1. Write a custom vLLM model definition for `Fuse3ForCausalLM`
-2. Register it with vLLM's model registry
-3. Handle the hybrid conv+attention layers and expert routing
+**Plugin repo**: [`Akahsizrr/fuse-1-Lite-vLLM`](https://huggingface.co/Akahsizrr/fuse-1-Lite-vLLM)
 
-Contributions welcome — see the model code in `fuse3_model.py` for the full architecture.
+```bash
+# Install the plugin
+pip install git+https://huggingface.co/Akahsizrr/fuse-1-Lite-vLLM
+
+# Serve with vLLM
+vllm serve Akahsizrr/fuse-1-Lite \
+  --mamba-cache-mode align \
+  --max-model-len 4096
+```
+
+```python
+from vllm import LLM
+
+llm = LLM(
+    model="Akahsizrr/fuse-1-Lite",
+    mamba_cache_mode="align",
+    max_model_len=4096,
+)
+output = llm.generate("Write a Python function to check if a number is prime.")
+```
+
+The plugin registers `Fuse3ForCausalLM` with vLLM's `ModelRegistry` via the
+`vllm.general_plugins` entry point. It reuses vLLM's native LFM2 attention
+and short-conv layers, adding the expert MoE block after each augmented
+layer's FFN.
 
 ### MLX (Apple Silicon)
 
-> **Note:** MLX does not currently support the custom `Fuse3ForCausalLM` architecture.
-> The model's hybrid conv+attention layers and MoE expert routing require a custom
-> MLX model implementation. Use transformers with MPS backend on Apple Silicon.
+fuse-1 Lite is available in MLX format for Apple Silicon (M1+).
+
+**MLX repo**: [`Akahsizrr/fuse-1-Lite-MLX`](https://huggingface.co/Akahsizrr/fuse-1-Lite-MLX)
 
 ```python
-# Run on Apple Silicon with MPS backend
-import torch
-from transformers import AutoModelForCausalLM, AutoTokenizer
+from mlx_lm import load, generate
 
-model = AutoModelForCausalLM.from_pretrained(
-    "Akahsizrr/fuse-1-Lite",
-    torch_dtype=torch.float16,
-    device_map="mps",
-    trust_remote_code=True,
+model, tokenizer = load("Akahsizrr/fuse-1-Lite-MLX", trust_remote_code=True)
+
+prompt = tokenizer.apply_chat_template(
+    [{"role": "user", "content": "Write a Python function to check if a number is prime."}],
+    tokenize=False, add_generation_prompt=True,
 )
-tokenizer = AutoTokenizer.from_pretrained("Akahsizrr/fuse-1-Lite")
+
+response = generate(model, tokenizer, prompt=prompt, max_tokens=512)
+print(response)
+```
+
+```bash
+# CLI
+mlx_lm.generate --model Akahsizrr/fuse-1-Lite-MLX --trust-remote-code --prompt "Write a Python fizzbuzz"
 ```
 
+The MLX model file (`fuse3_mlx.py`) extends MLX's native LFM2 implementation
+with the same expert MoE augmentation. It uses `model_file` in config.json
+with `trust_remote_code=True` for loading.
+
 ### GGUF / llama.cpp
 
-> **Note:** GGUF conversion is not currently supported. The custom `Fuse3ForCausalLM`
-> architecture is not recognized by llama.cpp's `convert_hf_to_gguf.py`. LFM2 itself
-> IS supported by llama.cpp (see `lfm2.cpp`), but the expert augmentation layers
-> require a custom llama.cpp model definition.
+fuse-1 Lite is available in GGUF format for llama.cpp.
+
+**GGUF repo**: [`Akahsizrr/fuse-1-Lite-GGUF`](https://huggingface.co/Akahsizrr/fuse-1-Lite-GGUF)
+
+> **Note:** The GGUF uses the custom `fuse3` architecture. Stock llama.cpp
+> cannot load it — you need a llama.cpp fork with Fuse3 support. The GGUF
+> repo includes the C++ graph builder (`src/models/fuse3.cpp`), Python
+> converter (`conversion/fuse3.py`), and integration guide (`INTEGRATION.md`).
+
+```bash
+# Build llama.cpp with Fuse3 support (see INTEGRATION.md in the GGUF repo)
+./llama-cli -m fuse-1-Lite-f16.gguf \
+  -p "Write a Python function to check if a number is prime." \
+  -n 512 --temp 0.1
+```
+
+The C++ implementation reuses LFM2's attention and short-conv graph builders,
+adding the expert MoE block (router → top-k → SwiGLU experts → scale → add)
+after each augmented layer's dense FFN.
 
-### Transformers (Recommended)
+### Transformers (Universal)
 
-The recommended way to run fuse-1 Lite is with transformers:
+The recommended way to run fuse-1 Lite on any platform:
 
 ```bash
 pip install transformers torch bitsandbytes accelerate
@@ -294,11 +341,15 @@ pip install transformers torch bitsandbytes accelerate
 
 ### VRAM Requirements
 
-| Precision | VRAM | Recommended GPU |
-|-----------|------|-----------------|
-| bfloat16 | ~12 GB | L4, A10G, RTX 4090 |
-| 8-bit | ~7 GB | T4, L4, RTX 3060 |
-| 4-bit | ~4.5 GB | T4, RTX 3060, M2 Pro |
+| Backend | Precision | VRAM/Memory | Recommended Hardware |
+|---------|-----------|-------------|---------------------|
+| Transformers | bfloat16 | ~12 GB | L4, A10G, RTX 4090 |
+| Transformers | 8-bit | 6.00 GB | T4, L4, RTX 3060 |
+| Transformers | 4-bit | 3.36 GB | T4, RTX 3060, M2 Pro |
+| vLLM | bfloat16 | ~12 GB | A10G, A100, H100 |
+| MLX | float16 | ~12 GB | M1 Pro+, M2, M3, M4 |
+| llama.cpp | F16 | ~11.4 GB | Any CPU/GPU |
+| llama.cpp | Q4_K_M | ~4 GB | Any CPU/GPU |
 
 ### Sample Outputs