---
base_model:
- FINAL-Bench/Darwin-4B-David
- Qwen/Qwen3.5-4B
language:
- ko
- en
- zh
- ja
- de
- fr
- es
license: apache-2.0
pipeline_tag: text-generation
library_name: transformers
tags:
- merge
- evolutionary-merge
- darwin
- darwin-v6
- model-mri
- cross-architecture
- ffn-crossbreed
- cma-es
- hybrid-vigor
- transformer-mamba
- reasoning
- gemma4
- qwen3.5
- gated-deltanet
- korean
- multilingual
- gpqa
- open-source
- world-first
model-index:
- name: Darwin-4B-Genesis
results:
- task:
type: text-generation
name: Korean Cultural Understanding
dataset:
name: CLIcK
type: EunsuKim/CLIcK
metrics:
- type: accuracy
value: 92.0
name: Accuracy
verified: false
- task:
type: text-generation
name: Multi-Step Reasoning
dataset:
name: MuSR
type: TAUR-Lab/MuSR
metrics:
- type: accuracy
value: 70.0
name: Accuracy
verified: false
---
# Darwin-4B-Genesis
Darwin-4B-Genesis is presented in the paper [Darwin Family: MRI-Trust-Weighted Evolutionary Merging for Training-Free Scaling of Language-Model Reasoning](https://arxiv.org/abs/2605.14386).
> **World's first Transformer × Mamba evolutionary cross-architecture FFN breeding** | CLIcK 92% | MuSR 70% | A 4B model outperforming 27B | CMA-ES 42-dimensional genome search | Hybrid Vigor demonstrated | Apache 2.0
---
## What Is This?
Darwin-4B-Genesis is the 3rd generation Darwin model and the **world's first model to successfully crossbreed FFN layers across different architectures** — Transformer (Gemma4) and Mamba (Qwen3.5 GatedDeltaNet) — using evolutionary optimization.
The father's Attention layers (Gemma4 Transformer) are preserved at 100%, while the mother's FFN knowledge (Qwen3.5 Mamba) is transplanted at layer-specific optimal ratios discovered automatically by CMA-ES across 42 dimensions.
The result: the child **outperforms both parents on every benchmark** — a phenomenon known as **Hybrid Vigor**.
---
## Why This Matters
### 1. World First
Existing hybrid models (Jamba, Nemotron-H, Granite 4.0) are all **designed and trained from scratch**. Darwin-4B-Genesis takes **two already-trained models** from different architecture families and breeds them evolutionarily — with **zero additional training**.
### 2. Hybrid Vigor Demonstrated
| Benchmark | David (Father) | Qwen3.5-4B (Mother) | **Genesis (Child)** |
|---|---|---|---|
| CLIcK | 90% | ~50% (est.) | **92%** ✅ |
| MuSR | 65% | ~55% (est.) | **70%** ✅ |
The child surpasses **both** parents. This is the first demonstration of Hybrid Vigor in AI model breeding.
---
## Benchmarks
| Benchmark | Genesis | David (Gen2) | K-AI #1 (27B) |
|---|---|---|---|
| **CLIcK** (Korean culture) | **92%** | 90% | 0.794 |
| **MuSR** (multi-step reasoning) | **70%** | 65% | 0.604 |
| **GPQA** (deep reasoning) | ~60% | ~60% | — |
---
## How It Works
### Cross-Architecture FFN Breeding
```
Father: Darwin-4B-David (Gemma4 Transformer, hidden=2560, 42 layers)
Mother: Qwen/Qwen3.5-4B (GatedDeltaNet/Mamba, hidden=2560, 32 layers)
Key insight: hidden_size matches (2560) → direct FFN replacement possible
Method: Attention 100% from Father, FFN blended at per-layer optimal ratios
Optimizer: CMA-ES (Covariance Matrix Adaptation Evolution Strategy)
Genome: 42 dimensions (one ratio per layer)
Fitness: CLIcK 60% + MuSR 40% composite score
Frozen layers: L15, L16, L22, L23, L24, L25 (Korean language preservation)
```
### Optimal Genome Discovered by CMA-ES
```
L00: 0.206 ██████████░ 21% Qwen
L07: 0.000 ░░░░░░░░░░░ Auto-protected by CMA-ES
L15: 0.000 ░░░░░░░░░░░ Frozen (Korean)
L22: 0.000 ░░░░░░░░░░░ Frozen (Korean)
L29: 0.291 ██████████████░ 29% Qwen (maximum)
L31: 0.244 ████████████░ 24% Qwen
L32: 0.273 █████████████░ 27% Qwen
```
Key finding: CMA-ES applied the **most aggressive Qwen blending to the final layers (L29-32)**, which govern output quality.
---
## Usage
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained(
"FINAL-Bench/Darwin-4B-Genesis",
trust_remote_code=True,
)
model = AutoModelForCausalLM.from_pretrained(
"FINAL-Bench/Darwin-4B-Genesis",
dtype="bfloat16",
device_map="auto",
trust_remote_code=True,
)
messages = [{"role": "user", "content": "Explain how hybrid vigor works in genetics."}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
print(tokenizer.decode(outputs[0][inputs['input_ids'].shape[-1]:], skip_special_tokens=True))
```
---
## Genealogy
```
google/gemma-4-E4B-it × TeichAI/Claude-Opus-Distill-E4B
→ Darwin-4B-Opus (Gen 1, DARE-TIES merge)
Darwin-4B-Opus × DavidAU/DECKARD-Expresso-Universe
→ Darwin-4B-David (Gen 2, MRI-guided merge, CLIcK 90%)
Darwin-4B-David × Qwen/Qwen3.5-4B
→ Darwin-4B-Genesis (Gen 3, Cross-Arch FFN Breeding, CLIcK 92%) ★
```
---
## Citation
```bibtex
@misc{vidraft_darwin_4b_genesis,
title = {Darwin-4B-Genesis: World's First Cross-Architecture FFN Breeding},
author = {VIDRAFT},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/FINAL-Bench/Darwin-4B-Genesis}}
}
@article{kim2026darwin,
title={Darwin Family: MRI-Trust-Weighted Evolutionary Merging for Training-Free Scaling of Language-Model Reasoning},
author={Kim, Taebong and Hong, Youngsik and Kim, Minsik and Choi, Sunyoung and Jang, Jaewon and Shin, Junghoon and Kim, Minseo},
journal={arXiv preprint arXiv:2605.14386},
year={2026}
}
```