Instructions to use darcar0/quotebound-27b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use darcar0/quotebound-27b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="darcar0/quotebound-27b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("darcar0/quotebound-27b", dtype="auto")

PEFT
How to use darcar0/quotebound-27b with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use darcar0/quotebound-27b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "darcar0/quotebound-27b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "darcar0/quotebound-27b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/darcar0/quotebound-27b

SGLang

How to use darcar0/quotebound-27b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "darcar0/quotebound-27b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "darcar0/quotebound-27b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "darcar0/quotebound-27b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "darcar0/quotebound-27b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use darcar0/quotebound-27b with Docker Model Runner:
```
docker model run hf.co/darcar0/quotebound-27b
```

quotebound-27b / benchmark_progression.svg

darcar0

Rebrand public release to Quotebound 27B

dab6650 verified about 2 months ago

preview code

raw

history blame contribute delete

7.03 kB

	<svg xmlns="http://www.w3.org/2000/svg" width="1160" height="612" viewBox="0 0 1160 612" fill="none">
	<rect width="1160" height="612" rx="28" fill="#fffaf2"/>
	<rect x="1" y="1" width="1158" height="610" rx="27" stroke="rgba(29,26,24,0.10)"/>
	<text x="64" y="52" font-family="Space Grotesk, sans-serif" font-size="28" font-weight="700" fill="#1d1a18">Benchmark progression on frozen probe_v0</text>
	<text x="64" y="78" font-family="IBM Plex Mono, monospace" font-size="12" letter-spacing="1.5" fill="#685e56">MAIN BENCHMARK-FACING STACKS</text>

	<g transform="translate(720,48)">
	<rect x="0" y="0" width="14" height="14" rx="7" fill="#1d1a18"/>
	<text x="22" y="11" font-size="12" fill="#685e56">task</text>
	<rect x="88" y="0" width="14" height="14" rx="7" fill="#ef7c30"/>
	<text x="110" y="11" font-size="12" fill="#685e56">strict</text>
	<rect x="182" y="0" width="14" height="14" rx="7" fill="#3b7f6a"/>
	<text x="204" y="11" font-size="12" fill="#685e56">evidence F1</text>
	<rect x="306" y="0" width="14" height="14" rx="7" fill="#7a5cff"/>
	<text x="328" y="11" font-size="12" fill="#685e56">quote F1</text>
	</g>

	<g transform="translate(64,98)">
	<rect x="0" y="0" width="1032" height="88" rx="22" fill="rgba(255,255,255,0.72)" stroke="rgba(29,26,24,0.08)"/>
	<text x="20" y="30" font-family="Space Grotesk, sans-serif" font-size="18" font-weight="600" fill="#1d1a18">Old v2 baseline</text>

	<g transform="translate(328,16)">
	<rect x="0" y="0" width="372" height="12" rx="6" fill="rgba(29,26,24,0.08)" />
	<rect x="0" y="0" width="355" height="12" rx="6" fill="#1d1a18" />
	<text x="384" y="10" font-size="12" fill="#685e56">task 0.9545</text>
	</g>


	<g transform="translate(328,34)">
	<rect x="0" y="0" width="372" height="12" rx="6" fill="rgba(29,26,24,0.08)" />
	<rect x="0" y="0" width="67" height="12" rx="6" fill="#ef7c30" />
	<text x="384" y="10" font-size="12" fill="#685e56">strict 0.1818</text>
	</g>


	<g transform="translate(328,52)">
	<rect x="0" y="0" width="372" height="12" rx="6" fill="rgba(29,26,24,0.08)" />
	<rect x="0" y="0" width="325" height="12" rx="6" fill="#3b7f6a" />
	<text x="384" y="10" font-size="12" fill="#685e56">evidence 0.8758</text>
	</g>


	<g transform="translate(328,70)">
	<rect x="0" y="0" width="372" height="12" rx="6" fill="rgba(29,26,24,0.08)" />
	<rect x="0" y="0" width="92" height="12" rx="6" fill="#7a5cff" />
	<text x="384" y="10" font-size="12" fill="#685e56">quote 0.2494</text>
	</g>

	</g>

	<g transform="translate(64,206)">
	<rect x="0" y="0" width="1032" height="88" rx="22" fill="rgba(255,255,255,0.72)" stroke="rgba(29,26,24,0.08)"/>
	<text x="20" y="30" font-family="Space Grotesk, sans-serif" font-size="18" font-weight="600" fill="#1d1a18">Bridge checkpoint-2</text>

	<g transform="translate(328,16)">
	<rect x="0" y="0" width="372" height="12" rx="6" fill="rgba(29,26,24,0.08)" />
	<rect x="0" y="0" width="372" height="12" rx="6" fill="#1d1a18" />
	<text x="384" y="10" font-size="12" fill="#685e56">task 1.0000</text>
	</g>


	<g transform="translate(328,34)">
	<rect x="0" y="0" width="372" height="12" rx="6" fill="rgba(29,26,24,0.08)" />
	<rect x="0" y="0" width="101" height="12" rx="6" fill="#ef7c30" />
	<text x="384" y="10" font-size="12" fill="#685e56">strict 0.2727</text>
	</g>


	<g transform="translate(328,52)">
	<rect x="0" y="0" width="372" height="12" rx="6" fill="rgba(29,26,24,0.08)" />
	<rect x="0" y="0" width="328" height="12" rx="6" fill="#3b7f6a" />
	<text x="384" y="10" font-size="12" fill="#685e56">evidence 0.8844</text>
	</g>


	<g transform="translate(328,70)">
	<rect x="0" y="0" width="372" height="12" rx="6" fill="rgba(29,26,24,0.08)" />
	<rect x="0" y="0" width="164" height="12" rx="6" fill="#7a5cff" />
	<text x="384" y="10" font-size="12" fill="#685e56">quote 0.4409</text>
	</g>

	</g>

	<g transform="translate(64,314)">
	<rect x="0" y="0" width="1032" height="88" rx="22" fill="rgba(255,255,255,0.72)" stroke="rgba(29,26,24,0.08)"/>
	<text x="20" y="30" font-family="Space Grotesk, sans-serif" font-size="18" font-weight="600" fill="#1d1a18">Bridge + deterministic_v2</text>

	<g transform="translate(328,16)">
	<rect x="0" y="0" width="372" height="12" rx="6" fill="rgba(29,26,24,0.08)" />
	<rect x="0" y="0" width="372" height="12" rx="6" fill="#1d1a18" />
	<text x="384" y="10" font-size="12" fill="#685e56">task 1.0000</text>
	</g>


	<g transform="translate(328,34)">
	<rect x="0" y="0" width="372" height="12" rx="6" fill="rgba(29,26,24,0.08)" />
	<rect x="0" y="0" width="152" height="12" rx="6" fill="#ef7c30" />
	<text x="384" y="10" font-size="12" fill="#685e56">strict 0.4091</text>
	</g>


	<g transform="translate(328,52)">
	<rect x="0" y="0" width="372" height="12" rx="6" fill="rgba(29,26,24,0.08)" />
	<rect x="0" y="0" width="328" height="12" rx="6" fill="#3b7f6a" />
	<text x="384" y="10" font-size="12" fill="#685e56">evidence 0.8844</text>
	</g>


	<g transform="translate(328,70)">
	<rect x="0" y="0" width="372" height="12" rx="6" fill="rgba(29,26,24,0.08)" />
	<rect x="0" y="0" width="214" height="12" rx="6" fill="#7a5cff" />
	<text x="384" y="10" font-size="12" fill="#685e56">quote 0.5773</text>
	</g>

	</g>

	<g transform="translate(64,422)">
	<rect x="0" y="0" width="1032" height="88" rx="22" fill="#fff3eb" stroke="rgba(29,26,24,0.08)"/>
	<text x="20" y="30" font-family="Space Grotesk, sans-serif" font-size="18" font-weight="600" fill="#1d1a18">Bridge + deterministic_v3</text>

	<g transform="translate(328,16)">
	<rect x="0" y="0" width="372" height="12" rx="6" fill="rgba(29,26,24,0.08)" />
	<rect x="0" y="0" width="372" height="12" rx="6" fill="#1d1a18" />
	<text x="384" y="10" font-size="12" fill="#685e56">task 1.0000</text>
	</g>


	<g transform="translate(328,34)">
	<rect x="0" y="0" width="372" height="12" rx="6" fill="rgba(29,26,24,0.08)" />
	<rect x="0" y="0" width="372" height="12" rx="6" fill="#ef7c30" />
	<text x="384" y="10" font-size="12" fill="#685e56">strict 1.0000</text>
	</g>


	<g transform="translate(328,52)">
	<rect x="0" y="0" width="372" height="12" rx="6" fill="rgba(29,26,24,0.08)" />
	<rect x="0" y="0" width="372" height="12" rx="6" fill="#3b7f6a" />
	<text x="384" y="10" font-size="12" fill="#685e56">evidence 1.0000</text>
	</g>


	<g transform="translate(328,70)">
	<rect x="0" y="0" width="372" height="12" rx="6" fill="rgba(29,26,24,0.08)" />
	<rect x="0" y="0" width="372" height="12" rx="6" fill="#7a5cff" />
	<text x="384" y="10" font-size="12" fill="#685e56">quote 1.0000</text>
	</g>

	</g>
	</svg>