Instructions to use alpha-ai/Deep-Reason-SMALL-V0-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use alpha-ai/Deep-Reason-SMALL-V0-GGUF with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("alpha-ai/Deep-Reason-SMALL-V0-GGUF", dtype="auto")

llama-cpp-python

How to use alpha-ai/Deep-Reason-SMALL-V0-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="alpha-ai/Deep-Reason-SMALL-V0-GGUF",
	filename="Deep-Reason-SMALL-V0.Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use alpha-ai/Deep-Reason-SMALL-V0-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf alpha-ai/Deep-Reason-SMALL-V0-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf alpha-ai/Deep-Reason-SMALL-V0-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf alpha-ai/Deep-Reason-SMALL-V0-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf alpha-ai/Deep-Reason-SMALL-V0-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf alpha-ai/Deep-Reason-SMALL-V0-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf alpha-ai/Deep-Reason-SMALL-V0-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf alpha-ai/Deep-Reason-SMALL-V0-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf alpha-ai/Deep-Reason-SMALL-V0-GGUF:Q4_K_M

Use Docker

docker model run hf.co/alpha-ai/Deep-Reason-SMALL-V0-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use alpha-ai/Deep-Reason-SMALL-V0-GGUF with Ollama:
```
ollama run hf.co/alpha-ai/Deep-Reason-SMALL-V0-GGUF:Q4_K_M
```

Unsloth Studio new

How to use alpha-ai/Deep-Reason-SMALL-V0-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for alpha-ai/Deep-Reason-SMALL-V0-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for alpha-ai/Deep-Reason-SMALL-V0-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for alpha-ai/Deep-Reason-SMALL-V0-GGUF to start chatting

Pi new

How to use alpha-ai/Deep-Reason-SMALL-V0-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf alpha-ai/Deep-Reason-SMALL-V0-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "alpha-ai/Deep-Reason-SMALL-V0-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use alpha-ai/Deep-Reason-SMALL-V0-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf alpha-ai/Deep-Reason-SMALL-V0-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default alpha-ai/Deep-Reason-SMALL-V0-GGUF:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use alpha-ai/Deep-Reason-SMALL-V0-GGUF with Docker Model Runner:
```
docker model run hf.co/alpha-ai/Deep-Reason-SMALL-V0-GGUF:Q4_K_M
```

Lemonade

How to use alpha-ai/Deep-Reason-SMALL-V0-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull alpha-ai/Deep-Reason-SMALL-V0-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Deep-Reason-SMALL-V0-GGUF-Q4_K_M

List all available models

lemonade list

Uploaded model

Developed by: alphaaico
License: apache-2.0
Finetuned from model : llama-3.2-3b-instruct-bnb-4bit

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

Deep-Reason-SMALL-V0

Overview Deep-Reason-SMALL-V0 is a fine-tuned version of llama-3.2-3b-instruct, designed for advanced reasoning and thinking capabilities. It has been trained using Reasoning GRPO techniques and a custom dataset curated for enhancing logical inference, decision-making, and structured reasoning.

Built with Unsloth and Hugging Face’s TRL, this model is optimized for faster inference and superior logical performance.

The model is available in GGUF and 16 Bit format and has been quantized to different levels to support various hardware configurations.

Model Details

Base Model: LLaMA-3 3B
Fine-tuned By: Alpha AI
Training Framework: Unsloth

Quantization Levels Available

q4_k_m
q5_k_m
q8_0
16 Bit (https://huggingface.co/alphaaico/Deep-Reason-SMALL-V0)

Key Features

Enhanced Reasoning: Fine-tuned using GRPO to improve problem-solving and structured thought processes.
Optimized for Thinking Tasks: Excels in logical, multi-step, and causal reasoning.
Structured XML Responses: Outputs are formatted using a structured reasoning-answer format for easy parsing. Outputs are formatted using structured <reasoning>...</think> and <answer>...</answer> sections for easy parsing.
Efficient Deployment: Available in GGUF format for local AI deployments on consumer hardware.

Response Format & Parsing Instructions Deep-Reason-SMALL-V0 follows a structured response format with designated XML-like tags for easy parsing. The XML responses will include tokens such as <reasoning>...</reasoning> and <answer>...</answer>. Users must extract the tokens accordingly when using programmatically. This ensures clarity and traceability in decision-making.

Ideal Configuration for using the GGUF Models

temperature = 0.8
top_p = 0.95
max_tokens = 1024
SYSTEM_PROMPT = """ Respond in the following format: <reasoning> ... </reasoning> <answer> ... </answer> """

Use Cases Deep-Reason-SMALL-V0 is best suited for:

Conversational AI – Improving chatbot and AI assistant reasoning.
AI Research – Studying logical thought modeling in AI.
Automated Decision Making – Use in AI-powered business intelligence systems.
Education & Tutoring – Helping students and professionals with structured learning.
Legal & Financial Analysis – Generating step-by-step arguments for case studies.

Limitations & Considerations

May require further fine-tuning for domain-specific logic.
Not a factual knowledge base – Focused on reasoning, not general knowledge retrieval.
Potential biases – Results depend on training data.
Computational Trade-off – Reasoning performance comes at the cost of slightly longer inference times.

License

This model is released under a permissible license.

Acknowledgments

Special thanks to the Unsloth team for providing an optimized training pipeline for LLaMA models.

Downloads last month: 33

GGUF

Model size

3B params

Architecture

llama

Hardware compatibility

4-bit

5-bit

8-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for alpha-ai/Deep-Reason-SMALL-V0-GGUF

Base model

meta-llama/Llama-3.2-3B-Instruct

Quantized

unsloth/Llama-3.2-3B-Instruct-bnb-4bit

Quantized

(113)

this model

Collection including alpha-ai/Deep-Reason-SMALL-V0-GGUF

Finetunes | SLMs and LLMs

Collection

Various variants of LLMs finetuned using proprietary data. • 27 items • Updated Jan 19 • 4