---
language:
  - en
  - code
license: apache-2.0
library_name: llama.cpp
base_model: Qwen/Qwen2.5-Coder-0.5B-Instruct
tags:
  - code
  - autocomplete
  - fim
  - fill-in-the-middle
  - python
  - qwen2.5-coder
  - minsore
  - gguf
pipeline_tag: text-generation
---

# 🟢 Quill 1 Preview

**Lightweight FIM autocomplete model for Python, built on Qwen2.5-Coder-0.5B.**

Quill is a 0.5B parameter code-completion model designed for **Fill-in-the-Middle (FIM)** tasks in IDEs and editors. It runs comfortably on consumer hardware and is one of the few publicly available 0.5B models trained specifically for autocomplete.

> Part of the **Minsore** family · [minsore.com](https://minsore.com)

---

## ✨ Highlights

- 🧠 **FIM-native** — understands `<fim_prefix>`, `<fim_suffix>`, `<fim_middle>`
- ⚡ **Real-time** — designed for live autocomplete in editors
- 📦 **Tiny** — 0.5B params, ~400 MB in Q4_K_M
- 🎯 **Purpose-built** — trained for autocomplete, not chat
- 🆓 **Apache 2.0** — same license as base model

---

![Quill comparison](assets/quill_comparison.png)

## 📊 Benchmarks

Evaluated against 0.5B–1B code models. All runs used `temperature=0.0`, raw FIM mode (`--chat-template none`).

| Benchmark | Quill 1 Preview | Arche-Codium 0.5B | Granite 4.0 1B | Qwen2.5-Coder 0.5B |
|---|---:|---:|---:|---:|
| **HumanEval@50** | 54.0% | **96.0%** | 58.0% | 52.0% |
| **HumanEval-Infilling** (EditSim) | **0.423** | 0.053 | 0.041 | 0.045 |
| **HumanEval-Infilling** (Exact Match) | **16%** | 0% | 0% | 0% |
| **NextBench v0.2** (autocomplete) | **45.0%** | 5.0% | 25.0% | 41.7% |
| **Delulu FIM** (EditSim) | **0.318** | 0.087 | 0.041 | 0.035 |

> 📌 **Key insight:** On FIM-specific benchmarks, Quill outperforms competitors by **8–10×**. On pure code generation (HumanEval), it matches the base model — training added FIM capability without hurting generation.

---

## 🚀 Quick Start

### llama.cpp

```bash
llama-server -m quill-1-preview.Q4_K_M.gguf \
  --port 8080 \
  -ngl 99 \
  --chat-template none
```

**Requirements:** any GPU with ≥2 GB VRAM (full offload), or partial CPU offload as fallback.

### FIM request

```bash
curl http://localhost:8080/infill \
  -H "Content-Type: application/json" \
  -d '{
    "input_prefix": "def add(a, b):\n    ",
    "input_suffix": "\n\nprint(add(2, 3))",
    "n_predict": 64,
    "temperature": 0.0
  }'
```

**Response:**

```json
{
  "content": "return a + b",
  "stop": true
}
```

### Python

```python
import requests

def fim_complete(prefix, suffix, max_tokens=64):
    r = requests.post("http://localhost:8080/infill", json={
        "input_prefix": prefix,
        "input_suffix": suffix,
        "n_predict": max_tokens,
        "temperature": 0.0,
        "stop": ["<fim_end>", "<|endoftext|>"],
    })
    return r.json()["content"]

print(fim_complete(
    "def fib(n):\n    if n <= 1:\n        return n\n    return ",
    "\n\nprint(fib(10))"
))
# → "fib(n - 1) + fib(n - 2)"
```

---

## ⚙️ Recommended Settings

| Parameter | Value | Notes |
|---|---|---|
| `n_predict` | 128–256 | Works best on short completions |
| `temperature` | 0.0 | Deterministic; for creative variation use 0.2 |
| `repeat_penalty` | 1.1 | Prevents number-looping |
| `stop` | `["<fim_end>", "<|endoftext|>", "\n\n\n"]` | Auto-stop on function boundary |
| `--chat-template` | `none` | **Required.** Quill is not a chat model |

---

## ⚠️ Limitations

- **Autocomplete only** — does not work as a chat assistant
- **Occasionally appends tests** after the function (inherited from mixed training data)
- **Number looping** — may repeat large integers on some prompts (use `repeat_penalty=1.1`)
- **1024 token context** — long files are truncated
- **Python-only** — trained on Python FIM data
- **Hallucinations** on unfamiliar APIs (EditSim 0.318 on Delulu FIM benchmark)

---

## 🧬 Training Details

| | |
|---|---|
| **Base model** | Qwen2.5-Coder-0.5B-Instruct |
| **Method** | QLoRA (r=16, α=16) |
| **Data** | Mixed FIM + Python instruction data |
| **Epochs** | 1 |
| **Context** | 1024 tokens |
| **FIM tokens** | `<fim_prefix>`, `<fim_suffix>`, `<fim_middle>`, `<fim_pad>` |

---

## 📁 Files

| File | Size | Description |
|---|---:|---|
| `quill-1-preview.Q4_K_M.gguf` | ~400 MB | Ready to use (recommended) |
| `quill-1-preview-f16.gguf` | ~1 GB | Full precision GGUF |
| `lora/` | ~35 MB | LoRA adapter for fine-tuning |

---

## 🗺️ Roadmap

- **Quill 2** — fix number looping, raise `pattern_hit` to 70%+
- **Quill 2** — multi-language support (JS/TS/Rust)
- **Pepper** — general-purpose code assistant (1.5B)
- **Symphony** — flagship code model (3B)

---

## 📜 License

Apache 2.0 — same as the base Qwen2.5-Coder-0.5B model.

---

## 🙏 Credits

- Base model: [Qwen2.5-Coder-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct) by Alibaba Cloud
- Training framework: [Unsloth](https://github.com/unslothai/unsloth)
- Inference: [llama.cpp](https://github.com/ggerganov/llama.cpp)

---

## 📬 Contact

**Minsore** — Ukrainian AI lab building open language models.

- 🌐 [minsore.com](https://minsore.com)
- 🤗 [huggingface.co/Minsore](https://huggingface.co/minsore)
- 💬 Built by [@Sollamon](https://huggingface.co/Sollamon)

---

<div align="center">

**⭐ If Quill is useful, star the repo and share your results.**

</div>