Upload README.md with huggingface_hub

README.md

@@ -7,14 +7,15 @@ tags:
 - icd-10
 - multi-label-classification
 - russian
-base_model: ai-forever/ruBert-base
+- conditional-distillation
+base_model: alexyalunin/RuBioBERT
 pipeline_tag: text-classification
 ---
 
-# ICD-10 subgroup classifier — group E (Russian)
+# ICD-10 subgroup classifier - group E (distilled specialist)
 
-Multi-label classifier over 3-character ICD-10 subgroups inside chapter **E**.  
-Fine-tuned from [`ai-forever/ruBert-base`](https://huggingface.co/ai-forever/ruBert-base) on Russian clinical text.
+Multi-label classifier over 3-character ICD-10 subgroups inside chapter **E**.
+This specialist was distilled from local BERT teacher models into `alexyalunin/RuBioBERT`. Teacher weights are not uploaded to Hugging Face.
 
 ## Intended use / Назначение
 - **EN:** Decision-support signal for suggesting candidate ICD-10 subgroups from Russian clinical notes. **Not** a substitute for clinician judgment; not validated for autonomous diagnosis.
@@ -23,29 +24,30 @@ Fine-tuned from [`ai-forever/ruBert-base`](https://huggingface.co/ai-forever/ruB
 ## Training data / Обучающие данные
 - Source CSV: `datasets/subgroups/group_E.csv`
 - SHA-256: `7bd98fc0eea937b8edf1391e86ca15afd2aed5c98996951f822684805713ed0b`
-- Produced by `ml/build_subgroup_datasets.ipynb` (iterative multi-label stratification by `parse_id`).
 - Splits: train=919 · val=200 · test=199
-- Labels: 45 (ordered, includes `E_OTHER` for rare codes collapsed during dataset build).
+- Labels: 45; rare/interface-only ids are listed in `label_map.json`.
 
-## Metrics (test split)
-| metric | value |
-|---|---|
-| macro_f1 | 0.5938 |
-| micro_f1 | 0.5619 |
-| weighted_f1 | 0.5713 |
-| subset_accuracy | 0.3568 |
-| hit@1 | 0.6432 |
-| hit@3 | 0.8141 |
-| recall@3 | 0.8047 |
-| mrr | 0.7459 |
+## Training route
+- Approach: `local_teacher_ensemble_knowledge_distillation`
+- Base model: `alexyalunin/RuBioBERT`
+- Direct validation hit@3: `0.835`
+- No-distillation threshold: `0.9`
+- Teacher models (fallback KD only): `['alexyalunin/RuBioRoBERTa', 'ai-forever/ruBert-base', 'DeepPavlov/rubert-base-cased']`
+- Selected KD config (fallback only): temperature=`2.0`, hard_loss_weight=`0.5`
 
-Full per-label breakdown in `metrics.json`.
+## Metrics (test split)
+| metric | final specialist | teacher ensemble / fallback |
+|---|---:|---:|
+| macro_f1 | 0.6122 | 0.6805 |
+| micro_f1 | 0.5606 | 0.6541 |
+| weighted_f1 | 0.5830 | 0.6549 |
+| subset_accuracy | 0.2965 | 0.4824 |
+| hit@1 | 0.6482 | 0.6784 |
+| hit@3 | 0.8241 | 0.8442 |
+| recall@3 | 0.8147 | 0.8382 |
+| mrr | 0.7516 | 0.7735 |
 
-## Limitations / Ограничения
-- Russian only; heavy reliance on clinical abbreviations (АД, ТТГ, УЗИ, etc.).
-- Training text had PII redacted (`*ДАТА*`, `*ГОРОД*`, ...); model may behave differently on non-redacted input.
-- Small chapters (train rows < 250) were trained with heavy regularization; some labels may have low support.
-- Rare labels without positives in train are kept in the label map (see `label_map.json → rare_label_ids`) for interface stability but will effectively never fire.
+Full per-label breakdown is available in `metrics.json`.
 
 ## Inference
 ```python
@@ -62,12 +64,9 @@ inp = tok(text, return_tensors="pt", truncation=True, max_length=512)
 with torch.no_grad():
     probs = torch.sigmoid(mdl(**inp).logits)[0]
 preds = [mdl.config.id2label[i] for i, p in enumerate(probs.tolist()) if p >= 0.5]
-top3 = sorted(
+top5 = sorted(
     [(mdl.config.id2label[i], p) for i, p in enumerate(probs.tolist())],
     key=lambda x: -x[1],
-)[:3]
-print(preds, top3)
+)[:5]
+print(preds, top5)
 ```
-
-## Citation / Ссылка
-Built as part of the `ai-app` ICD-10 classification pipeline. Upstream model: `ai-forever/ruBert-base` (ai-forever).