README.md

metadata

language:
  - ko
  - en
license: apache-2.0
library_name: transformers
pipeline_tag: token-classification
base_model: openai/privacy-filter
tags:
  - token-classification
  - ner
  - pii
  - privacy
  - pii-masking
  - korean
  - english
  - finance
  - customer-service
  - security
  - infrastructure
  - bioes
  - viterbi
  - mixture-of-experts
datasets:
  - BCCard/privacy-filter-openpii-masking
metrics:
  - precision
  - recall
  - f1

MoAI-Privacy-Filter

MoAI-Privacy-Filter is a Korean and English privacy-related entity detection model built by full fine-tuning openai/privacy-filter. It recognizes 29 entity types and emits 117 BIOES token classes. The training data emphasizes financial services and customer-service/VOC text while also covering identity, security, and infrastructure scenarios.

The model detects entity spans but does not decide how they should be masked or retained. Applications can apply their own handling policy to each predicted label. This distinction is especially important for PORT and ORGANIZATION, which are non-PII disambiguation labels included in the output taxonomy.

On held-out validation, strict micro F1 is 0.9824 for ko and 0.9708 for en. On an independently generated Golden Set, strict micro F1 is 0.9732 for ko and 0.9650 for en.

1. Model Summary

Item	Value
Model version	v3
Training dataset	BCCard/privacy-filter-openpii-masking v1
Base model	openai/privacy-filter
Architecture	Approximately 1.4B-parameter MoE, 8 layers, hidden size 640, 128 local experts, top-4 expert routing
Task	Token classification with BIOES span boundaries
Languages	Korean and English
Primary domains	Financial services, customer service/VOC, identity, security, and infrastructure
Entity labels	29
Output classes	117 - O plus four BIOES classes for each entity label
Training sequence limit	1024 tokens before special tokens
Artifact format	BF16 safetensors with 8 attention sinks tensors retained in FP32
Tested software	Transformers 5.13.1 and PyTorch 2.13.0
License	Apache 2.0

The model version and dataset version use independent version numbers. This model is v3 and was trained on dataset v1.

2. Label Taxonomy

Label	Definition
PERSON	Full personal name as one span.
RRN	Korean resident registration number.
FRN	Korean foreign resident registration number.
SSN	Social-security-number family inherited from ai4privacy SOCIALNUM; not limited to the US 9-digit form.
GENERIC_ID	Identity-card or tax identifier that cannot be assigned to a more specific country-level label.
CARD_NUMBER	Credit or debit card PAN.
ACCOUNT_NUMBER	Bank account number.
SECRET	Password, API key, access token, or similar authentication secret.
USER_ID	Online account or member identifier.
EMAIL	Email address, including intentionally obfuscated forms represented in the training data.
PHONE	Mobile or landline telephone number, including intentionally verbalized forms represented in the training data.
PASSPORT	Passport number.
DRIVER_LICENSE	Driver's license number.
ADDRESS	City, street, and building components represented as one address span.
ZIPCODE	Postal code kept separate from ADDRESS.
DATE	Date or time. Timezone-only strings are not included in this label.
CARD_EXPIRY	Payment-card expiration date.
CVC	Card verification code.
IPIN	Korean I-PIN identifier.
TRANSACTION_APPROVAL_ID	Payment authorization or transaction approval identifier.
BUSINESS_ID	Business registration number or merchant identifier.
VIRTUAL_CARD_NUMBER	Alternate or virtual card number.
CI	Korean identity-linkage information value.
IPADDRESS	IPv4 network address.
MACADDRESS	48-bit MAC address.
IMEI	Mobile-equipment identifier.
PORT	Network service port from 0 to 65535; a non-PII disambiguation label.
ORGANIZATION	Company, bank, hospital, or other organization name; a non-PII disambiguation label.
URL	Full web URL, including path, query, and fragment when present.

Each entity label has B-, I-, E-, and S- boundary classes. Together with O, the model therefore has 4 x 29 + 1 = 117 output classes. O means that the model predicts no taxonomy entity at that token; it does not guarantee that the surrounding text is non-sensitive.

3. Usage

import torch
from transformers import AutoModelForTokenClassification, AutoTokenizer

model_id = "BCCard/MoAI-Privacy-Filter"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForTokenClassification.from_pretrained(model_id)
model.eval()

text = "고객 모아이님(900101-1234569)께서 010-0000-0000로 연락 요청하셨습니다."
encoded = tokenizer(
    text,
    return_offsets_mapping=True,
    add_special_tokens=False,
    truncation=True,
    max_length=1024,
    return_tensors="pt",
)
offsets = encoded.pop("offset_mapping")[0].tolist()

with torch.no_grad():
    logits = model(**encoded).logits.float()

print(tuple(logits.shape))  # (1, sequence_length, 117)

Apply constrained BIOES Viterbi decoding to logits[0], then map the decoded token spans to the original text with offsets. Character-span records can then be represented in the following form.

[
  {'start': 3, 'end': 6, 'label': 'PERSON'},
  {'start': 8, 'end': 22, 'label': 'RRN'},
  {'start': 26, 'end': 39, 'label': 'PHONE'}
]

Character offsets use Python's half-open interval [start, end). A downstream application could render those spans as follows, but this replacement behavior is not part of the model.

고객 [PERSON]님([RRN])께서 [PHONE]로 연락 요청하셨습니다.

3.1. Decoding

The reported metrics use constrained Viterbi decoding over the BIOES transition grammar, followed by whitespace boundary refinement. Independent per-token argmax can emit invalid BIOES sequences and is not the reported decoding path.

The bundled viterbi_calibration.json contains six transition biases. The default operating point sets all biases to zero, so BIOES transition constraints remain active without an additional precision-recall adjustment. Convert logits to FP32 before decoding.

Use a decoder that implements this BIOES constraint contract. The upstream openai/privacy-filter project provides the reference implementation and decoding behavior on which this model is based.

For batches, use right padding and pass only input_ids and attention_mask to the model. Keep offset_mapping outside the model for character-span reconstruction. Inputs longer than 1024 tokens were not represented in the training regime and should be chunked with enough overlap for the target use case.

4. Training

4.1. Data

Dataset	Role	Size
BCCard/privacy-filter-openpii-masking v1	Training	57,851 rows
BCCard/privacy-filter-openpii-masking v1	Validation and checkpoint selection	14,524 rows
Independent non-public Golden Set	Final evaluation only	4,000 rows - ko 2,920 and en 1,080

The Golden Set was not used for training, checkpoint selection, or calibration. The training and validation data contain all 29 labels in both languages. English represents 25.74% of train and 26.03% of validation.

The dataset combines relabeled rows from ai4privacy/pii-masking-openpii-1.5m with Korean perturbation, English replay, and statically authored synthesis rows. Synthesis covers positive, confusion, hard-negative, weak-cue, long-context, and multi-label scenarios. The dataset is designed as synthetic training data and contains no operational customer records.

4.2. Procedure

Item	Value
Method	Full fine-tuning of all parameters, including experts and router
Classification head	33 base classes expanded to 117 classes through taxonomy-aware initialization
Loss	Token-level cross-entropy
Sequence length	Maximum 1024 tokens before special tokens
Batch	Per-device 16, effective 16 on one GPU
Learning rate	1e-4 with linear decay
Warmup	543 steps, approximately 3% of 18,080 total steps
Optimizer	Fused AdamW, weight decay 0.0, maximum gradient norm 1.0
Epochs	5, with the best checkpoint selected by validation strict micro F1
Precision	FP32 master parameters with BF16 autocast; MoE router and expert compute protected in FP32
Seed	42 for training and data sampling
Hardware	1 NVIDIA H100 80 GB GPU

The selected FP32 checkpoint was exported as a deployment artifact in BF16. Artifact validation found 132 BF16 tensors and 8 FP32 attention sinks tensors.

4.3. Included Files

File	Purpose
model.safetensors	BF16 deployment weights with FP32 attention sinks.
config.json	Architecture and 117-class label mapping.
tokenizer.json and tokenizer_config.json	Tokenizer files associated with the base model.
label-taxonomy.yaml	Label definitions and source-to-target mapping used for training.
viterbi_calibration.json	Constrained-decoding operating-point biases.

5. Evaluation

5.1. Setup

The validation split was used for checkpoint selection and experiment comparison. Final generalization was measured on a separately generated Golden Set containing weak-context entities, surface-similar decoys, label-confusion pairs, and boundary variants.

The headline metrics are language-slice strict micro Precision, Recall, and F1. A predicted entity is correct only when both its label and complete span boundary match the reference. Golden evaluation uses constrained Viterbi decoding and whitespace boundary refinement.

5.2. Results

Language	Validation P	Validation R	Validation F1	Golden P	Golden R	Golden F1	F1 Difference
ko	0.9829	0.9819	0.9824	0.9735	0.9729	0.9732	-0.92%p
en	0.9708	0.9708	0.9708	0.9652	0.9649	0.9650	-0.58%p

F1 Difference is Golden F1 minus Validation F1. The smaller Golden scores indicate a limited generalization decrease of 0.92 percentage points for Korean and 0.58 percentage points for English.

5.3. Error Characteristics

The aggregate results do not mean that every label performs equally. Error analysis of the English Golden slice shows the most visible weaknesses in ACCOUNT_NUMBER, ZIPCODE, PORT, and ORGANIZATION. Frequent confusion directions include PORT versus ZIPCODE, ACCOUNT_NUMBER versus BUSINESS_ID or IPIN, and ORGANIZATION versus PERSON.

These patterns are consistent with labels that share numeric shapes or require contextual role information. Downstream systems should evaluate per-label behavior on their own traffic, especially when label identity changes the handling action.

6. Intended Use

Suitable uses include:

• Detecting privacy-related entities before Korean or English text is sent to an LLM or another downstream service.
• Supporting offline privacy review of customer-service text, documents, email, and logs.
• Producing typed entity spans for an application-specific masking, routing, retention, or review policy.

The model is not a complete anonymization system, a legal-compliance guarantee, or a substitute for domain-specific review. It should not be used as the sole control for high-impact decisions. Applications remain responsible for deciding whether each detected label is masked, transformed, retained, or escalated.

7. Limitations

• Synthetic evaluation - Training, validation, and Golden data are synthetic. Performance on real customer text, OCR noise, slang, novel obfuscation, and unseen document structures has not been established.
• Label-specific variation - High aggregate F1 can hide weaker labels and confusion pairs. ACCOUNT_NUMBER, ZIPCODE, PORT, and ORGANIZATION require particular attention based on the current Golden analysis.
• Non-PII labels - PORT and ORGANIZATION are deliberately predicted even though they are not PII. Consumers must not assume that every non-O label requires the same action.
• Context and boundary sensitivity - Weak contextual evidence, shared numeric formats, and long entity boundaries can produce missed entities, boundary errors, or label swaps.
• Long inputs - The base architecture supports a larger context, but training examples were limited to 1024 tokens and the observed dataset maximum was 801 tokens. Longer inputs require separate validation and should normally be chunked.
• Registry-backed identifiers - Synthetic account, telephone, passport, user, and social-security values cannot be exhaustively checked against private issuance registries. Any coincidental match with a real value is unintended.
• Language and domain scope - Evaluation covers Korean and English with emphasis on financial, customer-service/VOC, identity, security, and infrastructure contexts. Other languages and domains are unsupported.

8. License, Attribution, and Citation

The model is released under the Apache 2.0 license. Its training dataset is released under CC BY 4.0 and is derived from ai4privacy/pii-masking-openpii-1.5m; follow the dataset card for its attribution requirements.

@misc{bccard2026moaiprivacyfilterv3,
  title        = {MoAI-Privacy-Filter v3: Korean and English Privacy-Related Entity Detection},
  author       = {BC Card},
  year         = {2026},
  howpublished = {https://huggingface.co/BCCard/MoAI-Privacy-Filter},
  note         = {Full fine-tune of openai/privacy-filter on BCCard/privacy-filter-openpii-masking v1}
}

Related resources:

• Base model: openai/privacy-filter.
• Training dataset: BCCard/privacy-filter-openpii-masking.
• Upstream dataset: ai4privacy/pii-masking-openpii-1.5m.

9. Disclaimer

This model is provided as is, without warranties of accuracy, completeness, non-infringement, or fitness for a particular purpose. Users are responsible for testing the model in their own environment and ensuring that its use complies with applicable laws, regulations, contractual obligations, and organizational policies.