EU PII Anonimization Multilingual Detector

A newer, better version is available: bardsai/eu-pii-anonimization-multilang-v2. On real web pages, scrapes, PDFs and invoices its strict F1 is 0.973 vs 0.885 for this model, and it is better on every public benchmark we tested (Gretel 0.692 vs 0.615, Nemotron-PII 0.933 vs 0.882, OpenPII 0.997 vs 0.928). Same label set; ONNX and INT8 weights included. New projects should use v2.

A multilingual PII detector built for EU compliance reality: 24 official languages, GDPR special categories, AI Act high-risk data - in one model.

Try it

🔗 Live demo in Browser

Why this exists

GDPR and the AI Act require you to detect and redact personal data across every language your users write in. Most open-source PII models were trained on English with a few translations bolted on, and they cover the basics — names, emails, phone numbers — while missing exactly the categories regulators care about: biometric data, genetic data, health information, political opinions, ethnic origin.

bardsai/eu-pii-anonimization-multilang is trained end-to-end on real multilingual data (not English-translated), covers 36 entity classes mapped to GDPR Article 9 special categories and AI Act high-risk identifiers, and ships with quantized ONNX weights so you can run it in production pipelines without GPU infrastructure.

What's different

Native multilingual training. Real text in EU languages. Performance on Polish, German, French, Italian, and Spanish is comparable to the English baseline.
GDPR special categories covered. Health, biometric, genetic, and other Article 9 entities that most OSS PII models skip entirely.
Production-ready. ONNX export and INT8 quantized weights included. Runs on CPU at latencies that work inside RAG ingestion or real-time redaction pipelines.

Who this is for

Compliance and privacy engineers at EU companies who need to:

Redact PII from documents, support tickets, emails, and chat logs before storage or analysis
Sanitize datasets before training, sharing, or moving across jurisdictions
Filter inputs to RAG pipelines and search indexes so personal data doesn't leak into prompts or logs
Build audit trails for what was redacted, when, and why

Quick start

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

model_name = "bardsai/eu-pii-anonimization-multilang"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForTokenClassification.from_pretrained(model_name)

text = "John Smith, passport AB123456, phone +48 123 456 789"
inputs = tokenizer(text, return_tensors="pt", truncation=True)

with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.argmax(outputs.logits, dim=-1)

tokens = tokenizer.convert_ids_to_tokens(inputs["input_ids"][0])
labels = [model.config.id2label[p.item()] for p in predictions[0]]

for token, label in zip(tokens, labels):
    if label != "O":
        print(label, token)

For production, use the quantized ONNX weights in onnx/model_quantized.onnx — same outputs, ~4x smaller, CPU-friendly latency.

Entity coverage

36 classes across eight families, with B-/I- BIO labeling:

Personal identity — names, dates of birth, national ID numbers
Contact and location — addresses, emails, phone numbers, geolocation
Official documents — passports, driver's licenses, tax IDs
Financial — IBAN, credit card, account numbers
Technical identifiers — IP addresses, MAC addresses, device IDs, usernames
Organization data — employer, institutional affiliations
Health, biometric, genetic (GDPR Art. 9) — medical conditions, biometric identifiers, genetic data
Special-category (GDPR Art. 9) — racial/ethnic origin, political opinions, religious beliefs, sexual orientation, trade union membership

Full label list in config.json (id2label / label2id).

Training data

The model was trained on a combination of public and proprietary data:

Public datasets

ai4privacy/pii-masking-openpii-1m — by Ai4Privacy
piimb/privy
nvidia/Nemotron-PII — by NVIDIA

Proprietary data

Internal dataset — an in-house, multilingual PII dataset curated by bards.ai, covering EU languages and GDPR special categories underrepresented in public corpora.
Augmented data — additional training examples generated through data augmentation to improve coverage of rarer entity classes, languages, and identifier formats.

Limitations

This is a model, not a compliance program. A few things to keep in mind:

Performance varies by language, domain, and input quality. OCR noise, code-switching, and unusual formatting will degrade recall.
Ambiguous mentions (a name that's also a place, an ID-shaped number that isn't an ID) need post-processing rules or human review.
Detection ≠ legal sufficiency. Use this to support a redaction workflow, not to replace your DPO's judgment.
Threshold tuning matters. The right operating point depends on whether you're optimizing for recall (compliance) or precision (data utility).

Files

model.safetensors — model weights
config.json — config and label mapping
tokenizer.json, tokenizer_config.json — tokenizer assets
onnx/model.onnx — ONNX export
onnx/model_quantized.onnx — INT8 quantized for CPU production
training_args.bin — training metadata

Citation

@misc{bards.ai_2026,
    author       = { bards.ai and Michał Swędrowski and Michał Pogoda-Rosikoń and Karol Samorański },
    title        = { eu-pii-anonimization-multilang (Revision 6de9f68) },
    year         = 2026,
    url          = { https://huggingface.co/bardsai/eu-pii-anonimization-multilang },
    doi          = { 10.57967/hf/8721 },
    publisher    = { Hugging Face }
}

About bards.ai

We build product ML for teams shipping AI to real users — RAG, agents, fine-tuned models, evals, and the unglamorous infrastructure that keeps them working. 16+ open models on Hugging Face, 10+ publications, production deployments at Comcast, Chili Piper, and Surfer SEO.

bards.ai