Instructions to use UnderTides/DeepSeek-V4.1-Flash-1017-heretic-lora-dev with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use UnderTides/DeepSeek-V4.1-Flash-1017-heretic-lora-dev with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("sudoingx/DeepSeek-V4.1-Flash-1017-EXL3-2.7bpw") model = PeftModel.from_pretrained(base_model, "UnderTides/DeepSeek-V4.1-Flash-1017-heretic-lora-dev") - Notebooks
- Google Colab
- Kaggle
DeepSeek V4.1 Flash 1017 EXL3 2.7 — Heretic LoRA dev
A rank-1 LoRA adapter for sudoingx/DeepSeek-V4.1-Flash-1017-EXL3-2.7bpw, built from 8,000 English and Chinese activation samples using a refusal-direction ablation method based on Heretic and pqhaz's V4.1 recipe.
This version expands the data and domain coverage of the original 2.7bpw adapter, doubling the capture set and organizing it into 17 domain groups. It uses the same rank-1, strength-2.5 recipe, with a direction freshly computed from the broader dataset.
| Property | Value |
|---|---|
| Original model | DeepSeek-V4.1-Flash-1017 |
| Quantized base | sudoingx EXL3 2.7bpw |
| Target modules | Attention output projection wo_b |
| Layers | 2–39, zero-based |
| LoRA rank / alpha | 1 / 1 |
| Projection strength | 2.5 |
| Factor precision | FP32 |
| Row normalization | Off |
Data
The capture set contains 4,000 refusal-side prompts and 4,000 ordinary prompts. Each side contains 2,000 English and 2,000 Chinese prompts. Separate tuning and holdout sets contain 400 and 600 prompts, respectively, with the same balance across sides and languages.
| Side | Language | Source | Capture prompts |
|---|---|---|---|
| Refusal | English | OpenSafetyLab/Salad-Data | 1,083 |
| Refusal | English | PKU-Alignment/PKU-SafeRLHF | 917 |
| Refusal | Chinese | thu-coai/Safety-Prompts | 2,000 |
| Ordinary | English | bench-llm/or-bench | 1,871 |
| Ordinary | English | HuggingFaceH4/no_robots | 129 |
| Ordinary | Chinese | m-a-p/COIG-CQIA | 1,590 |
| Ordinary | Chinese | Safety-Prompts, ordinary-intent selection | 410 |
Safety-Prompts contributes seven typical safety scenarios and six instruction-attack categories. OR-Bench adds ordinary requests with sensitive wording. COIG-CQIA contributes general questions, knowledge, writing, technical tasks, finance, medicine and reviews.
Sampling uses 17 domain groups: abuse and hate, discrimination, sexual content, physical violence, mental health, crime and public safety, weapons/drugs/biosecurity, cybersecurity, privacy and identity, finance and fraud, misinformation and politics, medicine and public health, copyright, ethics and manipulation, environment and animals, labor and trade, and general tasks.
Capture inputs consist of questions and required context, up to 512 tokens. Preparation uses cross-source lexical deduplication, grouped splits and sampling matched by domain, language and length, with seed 20261022. Task-form matching covers 6,684 of the 8,000 selected prompts. data_provenance.json records source revisions, selection counts, domain distributions and split hashes.
Method
- Capture: Run the TP2 base with thinking disabled and serial, uncached requests. Save the final prompt token's residual after mHC post-processing and before the next layer's Engram injection: 40 layers × four mHC streams × 5,120 BF16 values, in checksummed shards of 16 samples.
- Compute: After unloading the model, average the four mHC streams in FP32 and accumulate each side's mean in FP64 on CPU. Normalize the difference between the refusal-side and ordinary-side means, remove its component along the ordinary mean, and normalize again to obtain direction
d. - Export: Decode the effective quantized
wo_bweights block by block and construct the rank-1 factors for each selected layer:
u = normalize(mean_refusal - mean_ordinary)
g = normalize(mean_ordinary)
d = normalize(u - dot(u, g) * g)
A = d.T @ W # [1, 8192]
B = -2.5 * d # [5120, 1]
delta_W = B @ A
W_effective = W + delta_W
The adapter is computed analytically from the full 8,000-prompt set and the quantized base weights. Projection strength 2.5 is incorporated into the factors, with lora_alpha/r = 1. This release uses the original adapter's fixed projection settings.
Runtime validation
Validation covered 43 implementation tests and 21 functional scenarios across 62 HTTP requests. The scenarios exercise base/original/dev model selection, concurrent LoRA requests, prefix-cache isolation, Chat Completions and Responses, streaming, multi-turn conversations, tool calls and results, and single-image and eight-image inputs.
Loading
Runtime: the sudoingX/dgx-spark-deepseek-v41 vLLM engine with the included EXL3/multimodal LoRA compatibility patch, using TP2 across two DGX Sparks.
Download the adapter and apply the patch inside the runtime image:
hf download UnderTides/DeepSeek-V4.1-Flash-1017-heretic-lora-dev --local-dir ./heretic-lora-dev
python3 ./heretic-lora-dev/runtime/apply_runtime_lora.py \
--site-packages /usr/local/lib/python3.12/dist-packages --check
python3 ./heretic-lora-dev/runtime/apply_runtime_lora.py \
--site-packages /usr/local/lib/python3.12/dist-packages
Place the adapter at the same absolute path on both nodes. Add these options to the base model's launch command:
--enable-lora \
--max-loras 1 \
--max-cpu-loras 1 \
--max-lora-rank 8 \
--lora-target-modules wo_b \
--lora-modules '{"name":"deepseek-v4.1-flash-1017-heretic-dev","path":"/absolute/path/heretic-lora-dev","base_model_name":"deepseek-v4.1-flash-1017"}'
Select deepseek-v4.1-flash-1017-heretic-dev as the request's model ID. The base model and independently registered adapters share one loaded base. The adapter rank is 1; the runtime rank capacity is 8.
Files and licenses
adapter_model.safetensorsandadapter_config.json: PEFT adapter.recipe.json: base metadata, projection settings and weight hashes.data_provenance.json: dataset sources and sampling records.runtime/: LoRA compatibility patch, installer and source fingerprints.checksums.json: file checksums.
Model license: MIT. Runtime patches: Apache-2.0. Source datasets retain their respective licenses.
- Downloads last month
- 97
Model tree for UnderTides/DeepSeek-V4.1-Flash-1017-heretic-lora-dev
Base model
deepseek-ai/DeepSeek-V4.1-Flash-1017