Instructions to use labqoat/qev-0.8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use labqoat/qev-0.8b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="labqoat/qev-0.8b")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("labqoat/qev-0.8b")
model = AutoModelForMultimodalLM.from_pretrained("labqoat/qev-0.8b", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use labqoat/qev-0.8b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "labqoat/qev-0.8b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "labqoat/qev-0.8b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/labqoat/qev-0.8b

SGLang

How to use labqoat/qev-0.8b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "labqoat/qev-0.8b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "labqoat/qev-0.8b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "labqoat/qev-0.8b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "labqoat/qev-0.8b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use labqoat/qev-0.8b with Docker Model Runner:
```
docker model run hf.co/labqoat/qev-0.8b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

qev-0.8b: the tiny local decision model, by Labqoat

▶ Live demo · MLX · GGUF

qev-0.8b

A small local decision model. Give it a state (text, JSON or an image) and typed questions; it returns calibrated probabilities for every option in one forward pass. It does not generate text.

77.1% on Benchmark Heaven, first of 27 decision models around 1B parameters.
3.5 ms per request on one L40S; 1.3 GB of RAM on a Mac in 8-bit.
Same System One API (POST /v1/systemone) on NVIDIA, Apple silicon and llama.cpp.

Which version

You have	Use	Latency (p50, text)
NVIDIA GPU (recommended)	this repo: `qev.py` / `qev_server.py` (fast CUDA engine)	3.5 ms (L40S)
Apple silicon	qev-0.8b-MLX	109 ms (M4 Pro)
llama.cpp (CPU, Metal, CUDA, Vulkan)	qev-0.8b-GGUF: `llama-server -hf labqoat/qev-0.8b-GGUF:Q8_0`	~17 ms (M4 Pro)
Anything else	this repo: `qev.py` (transformers backend)	not measured

Quickstart

pip install -r requirements.txt -r requirements-cuda.txt   # drop the second file without an NVIDIA GPU

from qev import QEV

m = QEV.from_pretrained("labqoat/qev-0.8b")
m.decide(
    {"ticket": "My card was charged twice for the same order, please refund one."},
    {"dept":   {"type": "choice", "instructions": "Which team should handle this?",
                "criteria": {"billing": "payments and refunds", "tech": "bugs", "sales": None}},
     "urgent": {"type": "noul", "instructions": "Is this urgent?"},
     "anger":  {"type": "score", "instructions": "How upset is the customer?", "criteria": ["calm", "annoyed", "angry"]}})
# {"dept": {"choice": "billing", "probabilities": {"billing": 0.95, ...}}, "urgent": {"noul": 0.16}, "anger": {"score": 1.05, ...}}

Server, same wire format as System One:

python qev_server.py   # http://127.0.0.1:8017/v1/systemone
curl http://127.0.0.1:8017/v1/systemone -H 'Content-Type: application/json' \
  -d '{"state": {"ticket": "Refund please"}, "questions": {"urgent": {"type": "noul", "instructions": "Is this urgent?"}}}'

Images: put <image:NAME> in the state and pass images={"NAME": "shot.png"} (HTTP: "images": {name: base64}). Images are resized to at most 448×448 (≤196 tokens). Image support is zero-shot: the vision tower is the base model's.

Tips

Put the facts in the state and ask one short question per option.
Use per_question=True for image requests and for several related questions: it is more consistent and more accurate (JevBench coherence 64.5 → 71.2; JevVision 0.757 → 0.783) at some extra compute.
Act when it is confident; escalate when it is not.

Questions

type	asks	answer
`choice`	pick one named option (up to 255)	`choice`, `probabilities`, `confidence`
`noul`	yes / no	`noul` = P(yes)
`score`	a level on an ordered scale	`score` (expected level), `probabilities`

The prompt layout and answer format are decider-ai 1.9.0's. Probabilities are sharpened with per-type temperatures fitted on our own calibration split (choice 1.7, noul 0.9, score 1.25; in qev_config.json).

Benchmarks

Every model ran the same items and scoring on one L40S, in its own published runtime. Accuracy in %.

model	params	p50 ms	BH	BH ECE ↓	JevBench	suite	JevExtra	JevSpatial	Rules-7
qev-0.8b	0.8B	3.5	77.1	0.063	85.1	61.7	72.0	74.2	63.7
decision-2b	2B	81.3	76.2	0.042	86.0	60.9	64.1	69.5	58.1
decider-2b	1.9B	7.2	75.8	0.065	85.9	70.7	58.8	–	–
EXAONE-4.0-1.2B-JEV	1.2B	27.8	68.8	0.097	83.9	67.3	64.0	70.0	58.8
malkuth-2b	2B	46.2	68.8	0.131	84.8	66.6	63.6	69.9	63.7

BH = Benchmark Heaven public (231 items, ±2.8 points). Suite = 22 public-dataset tasks. JevExtra, JevSpatial and Rules-7 are our own skill benchmarks (agents and safety; games and space; policy rules). Image input: 0.759 on our JevVision v0 benchmark, against 0.651 for the base model.

How to read these numbers. No benchmark item was used for training or for calibration, and the training data was screened against every benchmark item (no shared 16-token sequence with BH or JevBench). However:

benchmark scores were used to choose between checkpoints and weight soups, so they are not untouched held-out numbers;
part of the training data is written in the same formats as some benchmarks (for example long policy documents with amendments, sentiment scales, NLI, intent), because those are the formats the model is meant to serve;
JevExtra's indirect prompt-injection items share text with training rows, and its other skills share sources with training data.

Weak spots: 4-level score questions (JevBench score accuracy 0.39, mostly off by one), long or very hard inputs (BH long 0.51), choosing "cannot answer" / "no tool", and reading game boards or counting objects in images.

Training

Full fine-tune of Qwen3.5-0.8B-Base (dc7cdfe2) on about 2.4B tokens of decision data: a broad pass over 1.28M requests, anneals on newer data, a consistency pass (twin sets that must agree under rephrasing, reordering and complements), a weak-skill pass, and a games / rules / spatial pass. The release is a weight soup of two of these checkpoints.

Data: public datasets (about half the rows, with their own labels), program-generated data (about a third, exact labels), and rows written or labelled by larger models (the rest). Most rows are permissively licensed; about 10% are share-alike and under 1% research-use only. Each row records its source, license and how it was labelled. Whether those terms affect your use of the weights is for you to assess.

Limitations

score outputs are the weakest; treat them as soft signals.
Mostly English business, operations and agent decisions; other languages and domains are not evaluated.
Calibration was fitted on our data; recalibrate on your own labels when probabilities drive decisions.
It reads in one pass and does no reasoning; long, hard inputs are weaker.
Do not use it as the only decision maker for consequential decisions about people.

Files

model.safetensors (bf16 weights) · tokenizer and config files · qev_config.json (temperatures, option letters) · qev.py (runner) · qev_server.py (HTTP server) · qev_fast/ (CUDA engine) · qev_image.py, qev_image_server.py (transformers image path) · SHA256SUMS.

This release is the checkpoint called SG50 in our reports. License: Apache-2.0, as the base model.