DeepSeek V4.1 Flash 1017 EXL3 2.7 — Heretic LoRA dev

A rank-1 LoRA adapter for sudoingx/DeepSeek-V4.1-Flash-1017-EXL3-2.7bpw, built from 8,000 English and Chinese activation samples using a refusal-direction ablation method based on Heretic and pqhaz's V4.1 recipe.

This version expands the data and domain coverage of the original 2.7bpw adapter, doubling the capture set and organizing it into 17 domain groups. It uses the same rank-1, strength-2.5 recipe, with a direction freshly computed from the broader dataset.

Property Value
Original model DeepSeek-V4.1-Flash-1017
Quantized base sudoingx EXL3 2.7bpw
Target modules Attention output projection wo_b
Layers 2–39, zero-based
LoRA rank / alpha 1 / 1
Projection strength 2.5
Factor precision FP32
Row normalization Off

Data

The capture set contains 4,000 refusal-side prompts and 4,000 ordinary prompts. Each side contains 2,000 English and 2,000 Chinese prompts. Separate tuning and holdout sets contain 400 and 600 prompts, respectively, with the same balance across sides and languages.

Side Language Source Capture prompts
Refusal English OpenSafetyLab/Salad-Data 1,083
Refusal English PKU-Alignment/PKU-SafeRLHF 917
Refusal Chinese thu-coai/Safety-Prompts 2,000
Ordinary English bench-llm/or-bench 1,871
Ordinary English HuggingFaceH4/no_robots 129
Ordinary Chinese m-a-p/COIG-CQIA 1,590
Ordinary Chinese Safety-Prompts, ordinary-intent selection 410

Safety-Prompts contributes seven typical safety scenarios and six instruction-attack categories. OR-Bench adds ordinary requests with sensitive wording. COIG-CQIA contributes general questions, knowledge, writing, technical tasks, finance, medicine and reviews.

Sampling uses 17 domain groups: abuse and hate, discrimination, sexual content, physical violence, mental health, crime and public safety, weapons/drugs/biosecurity, cybersecurity, privacy and identity, finance and fraud, misinformation and politics, medicine and public health, copyright, ethics and manipulation, environment and animals, labor and trade, and general tasks.

Capture inputs consist of questions and required context, up to 512 tokens. Preparation uses cross-source lexical deduplication, grouped splits and sampling matched by domain, language and length, with seed 20261022. Task-form matching covers 6,684 of the 8,000 selected prompts. data_provenance.json records source revisions, selection counts, domain distributions and split hashes.

Method

  1. Capture: Run the TP2 base with thinking disabled and serial, uncached requests. Save the final prompt token's residual after mHC post-processing and before the next layer's Engram injection: 40 layers × four mHC streams × 5,120 BF16 values, in checksummed shards of 16 samples.
  2. Compute: After unloading the model, average the four mHC streams in FP32 and accumulate each side's mean in FP64 on CPU. Normalize the difference between the refusal-side and ordinary-side means, remove its component along the ordinary mean, and normalize again to obtain direction d.
  3. Export: Decode the effective quantized wo_b weights block by block and construct the rank-1 factors for each selected layer:
u = normalize(mean_refusal - mean_ordinary)
g = normalize(mean_ordinary)
d = normalize(u - dot(u, g) * g)

A = d.T @ W             # [1, 8192]
B = -2.5 * d            # [5120, 1]
delta_W = B @ A
W_effective = W + delta_W

The adapter is computed analytically from the full 8,000-prompt set and the quantized base weights. Projection strength 2.5 is incorporated into the factors, with lora_alpha/r = 1. This release uses the original adapter's fixed projection settings.

Runtime validation

Validation covered 43 implementation tests and 21 functional scenarios across 62 HTTP requests. The scenarios exercise base/original/dev model selection, concurrent LoRA requests, prefix-cache isolation, Chat Completions and Responses, streaming, multi-turn conversations, tool calls and results, and single-image and eight-image inputs.

Loading

Runtime: the sudoingX/dgx-spark-deepseek-v41 vLLM engine with the included EXL3/multimodal LoRA compatibility patch, using TP2 across two DGX Sparks.

Download the adapter and apply the patch inside the runtime image:

hf download UnderTides/DeepSeek-V4.1-Flash-1017-heretic-lora-dev --local-dir ./heretic-lora-dev
python3 ./heretic-lora-dev/runtime/apply_runtime_lora.py \
  --site-packages /usr/local/lib/python3.12/dist-packages --check
python3 ./heretic-lora-dev/runtime/apply_runtime_lora.py \
  --site-packages /usr/local/lib/python3.12/dist-packages

Place the adapter at the same absolute path on both nodes. Add these options to the base model's launch command:

--enable-lora \
--max-loras 1 \
--max-cpu-loras 1 \
--max-lora-rank 8 \
--lora-target-modules wo_b \
--lora-modules '{"name":"deepseek-v4.1-flash-1017-heretic-dev","path":"/absolute/path/heretic-lora-dev","base_model_name":"deepseek-v4.1-flash-1017"}'

Select deepseek-v4.1-flash-1017-heretic-dev as the request's model ID. The base model and independently registered adapters share one loaded base. The adapter rank is 1; the runtime rank capacity is 8.

Files and licenses

  • adapter_model.safetensors and adapter_config.json: PEFT adapter.
  • recipe.json: base metadata, projection settings and weight hashes.
  • data_provenance.json: dataset sources and sampling records.
  • runtime/: LoRA compatibility patch, installer and source fingerprints.
  • checksums.json: file checksums.

Model license: MIT. Runtime patches: Apache-2.0. Source datasets retain their respective licenses.

Downloads last month
97
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for UnderTides/DeepSeek-V4.1-Flash-1017-heretic-lora-dev

Datasets used to train UnderTides/DeepSeek-V4.1-Flash-1017-heretic-lora-dev