Text Classification
PEFT
Safetensors
Transformers
feedback-detection
user-satisfaction
lora
modernbert
mmbert
32k-context
Eval Results (legacy)
Instructions to use vllm-sr/mmbert32k-feedback-detector-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use vllm-sr/mmbert32k-feedback-detector-lora with PEFT:
from peft import PeftModel from transformers import AutoModelForSequenceClassification base_model = AutoModelForSequenceClassification.from_pretrained("vllm-sr/mmbert-32k-yarn") model = PeftModel.from_pretrained(base_model, "vllm-sr/mmbert32k-feedback-detector-lora") - Transformers
How to use vllm-sr/mmbert32k-feedback-detector-lora with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="vllm-sr/mmbert32k-feedback-detector-lora")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("vllm-sr/mmbert32k-feedback-detector-lora", device_map="auto") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from vllm-sr/mmbert32k-feedback-detector-lora: direct link, hf CLI and curl.
- Browser
- Download file 4.89 kB
-
https://huggingface.co/vllm-sr/mmbert32k-feedback-detector-lora/resolve/main/README.md
- Command line
-
hf download hf://vllm-sr/mmbert32k-feedback-detector-lora/README.md
-
curl -L -o README.md https://huggingface.co/vllm-sr/mmbert32k-feedback-detector-lora/resolve/main/README.md
4.89 kB
metadata
base_model: vllm-sr/mmbert-32k-yarn
library_name: peft
license: apache-2.0
language:
- en
- zh
- fr
- es
- multilingual
tags:
- text-classification
- feedback-detection
- user-satisfaction
- lora
- transformers
- modernbert
- mmbert
- 32k-context
datasets:
- vllm-sr/feedback-detector-dataset
metrics:
- accuracy
- f1
pipeline_tag: text-classification
model-index:
- name: mmbert32k-feedback-detector-lora
results:
- task:
type: text-classification
name: User Feedback Classification
dataset:
name: feedback-detector-dataset
type: vllm-sr/feedback-detector-dataset
metrics:
- name: Accuracy
type: accuracy
value: 0.9883
- name: F1 (macro)
type: f1
value: 0.9824
mmBERT-32K Feedback Detector (LoRA)
A 4-class user feedback classifier fine-tuned from mmbert-32k-yarn using LoRA (Low-Rank Adaptation).
Model Description
This model classifies user messages into 4 feedback categories to help conversational AI systems understand user satisfaction and respond appropriately:
| Label | ID | Description |
|---|---|---|
| SAT | 0 | User is satisfied with the response |
| NEED_CLARIFICATION | 1 | User needs more explanation or details |
| WRONG_ANSWER | 2 | User indicates the response was incorrect |
| WANT_DIFFERENT | 3 | User wants an alternative approach/answer |
Performance
Validation Results (2,985 samples):
| Metric | Value |
|---|---|
| Accuracy | 98.83% |
| F1 (macro) | 98.24% |
| F1 (weighted) | 98.83% |
Per-Class Performance:
| Class | Precision | Recall | F1-Score | Support |
|---|---|---|---|---|
| SAT | 1.0000 | 1.0000 | 1.0000 | 1,491 |
| NEED_CLARIFICATION | 0.9980 | 0.9980 | 0.9980 | 498 |
| WRONG_ANSWER | 0.9604 | 0.9739 | 0.9671 | 498 |
| WANT_DIFFERENT | 0.9715 | 0.9578 | 0.9646 | 498 |
Usage
With PEFT (Recommended)
from transformers import AutoModelForSequenceClassification, AutoTokenizer
from peft import PeftModel
# Load base model
base_model = AutoModelForSequenceClassification.from_pretrained(
"vllm-sr/mmbert-32k-yarn",
num_labels=4
)
tokenizer = AutoTokenizer.from_pretrained("vllm-sr/mmbert-32k-yarn")
# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "vllm-sr/mmbert32k-feedback-detector-lora")
model.eval()
# Inference
labels = ["SAT", "NEED_CLARIFICATION", "WRONG_ANSWER", "WANT_DIFFERENT"]
text = "I don't understand your explanation, can you clarify?"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
outputs = model(**inputs)
prediction = outputs.logits.argmax(-1).item()
print(f"Feedback: {labels[prediction]}") # Output: NEED_CLARIFICATION
Using Merged Model (No PEFT required)
For easier deployment, use the merged version:
from transformers import AutoModelForSequenceClassification, AutoTokenizer
model = AutoModelForSequenceClassification.from_pretrained(
"vllm-sr/mmbert32k-feedback-detector-merged"
)
tokenizer = AutoTokenizer.from_pretrained(
"vllm-sr/mmbert32k-feedback-detector-merged"
)
Training Details
Hyperparameters
| Parameter | Value |
|---|---|
| Base Model | vllm-sr/mmbert-32k-yarn |
| LoRA Rank | 64 |
| LoRA Alpha | 128 |
| LoRA Dropout | 0.1 |
| Target Modules | attn.Wqkv, attn.Wo, mlp.Wi, mlp.Wo |
| Learning Rate | 2e-5 |
| Batch Size | 16 |
| Epochs | 10 (early stopping at ~5.4) |
| Warmup Ratio | 0.1 |
| Weight Decay | 0.01 |
| Precision | bf16 |
| Optimizer | AdamW |
Training Data
Trained on vllm-sr/feedback-detector-dataset:
- Training samples: 17,896 (balanced across 4 classes)
- Validation samples: 2,985
Hardware
- GPU: AMD Instinct MI300X (192GB HBM3)
- Training Time: ~10 minutes
- Framework: PyTorch 2.x with ROCm
Multilingual Support
The model inherits multilingual capabilities from mmbert-32k-yarn (Glot500 tokenizer supporting 1800+ languages). Best performance on:
- English (primary)
- Chinese (Simplified/Traditional)
- French
- Spanish
Limitations
- The SAT class has the strongest performance; some edge cases between WRONG_ANSWER and WANT_DIFFERENT may be ambiguous
- Phrases like "That's perfect, no more questions" may sometimes be misclassified
- Best suited for conversational AI feedback detection, not general sentiment analysis
Citation
@misc{mmbert32k-feedback-detector,
title={mmBERT-32K Feedback Detector},
author={LLM Semantic Router Team},
year={2026},
publisher={Hugging Face},
url={https://huggingface.co/vllm-sr/mmbert32k-feedback-detector-lora}
}
License
Apache 2.0
Framework Versions
- PEFT: 0.18.1
- Transformers: 4.48+
- PyTorch: 2.6+