🕉️ Rudra — Agentic Fine-Tune of Qwen3.5-4B

Rudra is an autonomous, tool-using coding & file-operation agent. Built by Vyomma Intelligence · brain behind Rudra: Ritik Suman.

---

📌 What is Rudra?

rudra is a LoRA fine-tune of Qwen3.5-4B (Unsloth base) trained to run autonomous agent loops with strict, machine-parsable JSON envelopes. It is optimized for agentic workflows, function calling, tool use, structured output, and anti-hallucination behavior on consumer hardware (runs in a 6 GB GPU / RTX 3050 laptop).

Unlike chat models forced into JSON via few-shot prompts, Rudra is natively trained to emit a strict envelope:

{
  "plan": "…",
  "steps": ["…", "…"],
  "done": false
}

Every step is a tool invocation (read_file, write_file, list_files, …), and Rudra verifies every action with a tool result before claiming success — no phantom file edits, no invented outputs.

✨ Highlights

• 🔒 Strict JSON envelopes — valid_json = 1.0 across all 10 benchmark scenarios
• 🛠 100% tool-call accuracy — tool_accuracy = 1.0, natively fine-tuned for function-calling syntax
• 🚫 0% drift across 55+ agent-loop steps — the envelope never breaks mid-run
• ✅ Phantom-proof — never claims an action succeeded without tool evidence (the exact failure mode that plagues generic LLM agents)
• ⚡ Small & fast — 4B params, Q8_0 GGUF = 4.5 GB, runs fully in VRAM on RTX 3050 6 GB, O(moves) agent completion in ≤6-step budgets
• 🧩 Drop-in for Ollama — import the GGUF, wire it into your agent harness, done

🎯 Training

Base model	unsloth/Qwen3.5-4B (Q4_K_M GGUF serve path)
Method	LoRA fine-tune (PEFT) — r=16, α=32, merged into full weights, then re-quantized
Objective	Strict agent-envelope JSON · tool-call syntax · verification-before-claim discipline
Inference	Ollama / llama.cpp (GGUF Q8_0 canonical) · parameters: top_k 40, top_p 0.95, temp 0.4, repeat_penalty 1.1, ctx 8192

📊 Rudra v5 → v6: measured comparison (this release vs the previous one)

All numbers are from the same reproducible test harness — structured eval (eval_rudra, 10 scenarios, temp 0) and the Agentic Loop Battery (5 real file-operation missions × 2 runs @ temp 0.4, 6-step budget per turn, pass = tool-verified mission completion).

Metric	Rudra v5 (previous)	Rudra v6 (this release)
valid_json rate	1.00	1.00
tool-call accuracy	0.95	1.00
task completion	1.00	1.00
envelope drift (broken JSON mid-run)	0	0
Agent-loop battery passes	4/10	5/10
Envelope breaks in battery	0	0
Tests rewritten (suspicious passes)	0	0
Avg steps per run	—	4.5 (budget: 6)
phantom_probe (anti-hallucination trap)	0.83 verified	1.00 verified
Multi-file rename · report · recovery · web-research scenarios	✅ pass	✅ all pass with full verification

What changed from v5 → v6:

1. Perfect tool-calling — the last remaining 5% of tool-call misses (0.95 → 1.00) is gone.
2. Best-ever agent-loop score — 5/10 real missions completed under a strict 6-step budget, up from 4/10.
3. Full phantom-proofing — the phantom_probe trap (asking about non-existent files) now passes with verified evidence 100% of the time (was 0.83).
4. Cleaner merge path — v6 weights are merged and re-quantized directly (Q8_0), eliminating the quantization-path loss the previous release inherited.

Historical note: intermediate builds (v7 / v8) measured below v5 and are superseded — v8 scored 1/10 on the same battery and 0.90 tool-accuracy; it never shipped.

🚀 Quickstart (Ollama)

# create a Modelfile pointing at rudra_v6_q8.gguf, then:
ollama create rudra -f Modelfile
ollama run rudra

System prompt that makes Rudra behave like an agent:

You are Rudra, an autonomous coding agent.
1. Plan first; break the task into explicit steps before acting.
2. Use tools to inspect and change the workspace; never guess file contents you can read.
3. Verify after every change; never claim success without a tool result proving it.
4. If a tool fails, read the error, adjust the arguments, and retry once; never complain.
5. Keep long missions on track: re-state the remaining steps as you go.
6. Anti-hallucination: never invent file contents, outputs, or test results.
7. Finish with task_done and a concise summary.

Useful parameters: top_k 40 · top_p 0.95 · temperature 0.4 · repeat_penalty 1.1 · num_ctx 8192

🔬 Benchmark methodology (reproducibility)

• eval_rudra: 10 scenarios — single_write, rename_two, missing_recover, data_report, web_research, memory_use, phantom_probe, abs_path_trap, multi_read_summary, long_mission — temp 0 JSON-mode, each graded for valid_json, tool-hit, verification, task completion, drift.
• Agentic Loop Battery: real filesystem missions × 2 repeats @ temp 0.4 with a hard 6-step budget — counts envelope breaks, test-rewrites (a proxy for gaming the grader), step efficiency, and exploration overhead.

📦 Files

File	Purpose
rudra_v6_q8.gguf	Canonical Q8_0 GGUF — use this with Ollama / llama.cpp
tokenizer.json · tokenizer_config.json · chat_template.jinja · generation_config.json	Full tokenizer + chat template (matches the training format)

🏷️ Intended use

• Autonomous file-operation and coding assistants running locally (privacy-first)
• Agent frameworks that need guaranteed-parseable step envelopes
• Edge / consumer-GPU deployments where 4B-class agentic models shine

⚠️ Limitations

• Specialized for the agent-envelope + file/multi-tool domain — general chit-chat is out-of-scope by design
• Best results use the exact system prompt and parameters above
• 8192 default context; raise num_ctx only if your VRAM allows

---

Built by Vyomma Intelligence · Brain behind Rudra: Ritik Suman
Keywords: agentic AI · autonomous agent · function calling · tool use · JSON structured output · Qwen3.5-4B fine-tune · LoRA · GGUF · Ollama · llama.cpp · local LLM · 4B model · coding agent · anti-hallucination

If Rudra helps you build something, ⭐ the repo — it helps a lot.