| --- |
| base_model: PaddlePaddle/PaddleOCR-VL |
| library_name: peft |
| license: apache-2.0 |
| pipeline_tag: image-text-to-text |
| language: |
| - pl |
| tags: |
| - ocr |
| - lora |
| - transformers |
| - polish |
| - document-ai |
| - vision-language |
| datasets: |
| - synthetic-polish-ocr |
| --- |
| |
| # RysOCR - Polish OCR LoRA for PaddleOCR-VL |
|
|
| A LoRA adapter fine-tuned on PaddleOCR-VL specifically for **Polish text recognition**, with emphasis on correct handling of Polish diacritics (ą, ć, ę, ł, ń, ó, ś, ź, ż). |
|
|
| ## Motivation |
|
|
| Polish is underrepresented in OCR training data. Most vision-language OCR models struggle with Polish diacritics, often substituting: |
| - `ą` → `a` |
| - `ę` → `e` |
| - `ł` → `l` or `t` |
| - `ó` → `o` |
| - etc. |
|
|
| This model addresses that gap by fine-tuning on synthetic Polish document images covering addresses, invoices, receipts, names, and common phrases. |
|
|
| ## Model Details |
|
|
| | Property | Value | |
| |----------|-------| |
| | Base Model | [PaddlePaddle/PaddleOCR-VL](https://huggingface.co/PaddlePaddle/PaddleOCR-VL) | |
| | Method | LoRA (Low-Rank Adaptation) | |
| | LoRA Rank | 16 | |
| | LoRA Alpha | 32 | |
| | Target Modules | q_proj, k_proj, v_proj, o_proj | |
| | Training Framework | PEFT 0.18.0 + Transformers | |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoProcessor |
| from peft import PeftModel |
| from PIL import Image |
| |
| # Load base model |
| base_model = AutoModelForCausalLM.from_pretrained( |
| "PaddlePaddle/PaddleOCR-VL", |
| trust_remote_code=True, |
| torch_dtype="auto", |
| device_map="auto" |
| ) |
| |
| # Load LoRA adapter |
| model = PeftModel.from_pretrained(base_model, "anon13370/RysOCR") |
| |
| processor = AutoProcessor.from_pretrained( |
| "anon13370/RysOCR", |
| trust_remote_code=True |
| ) |
| |
| # Run inference |
| image = Image.open("your_document.png") |
| prompt = "OCR: " |
| |
| inputs = processor(images=image, text=prompt, return_tensors="pt") |
| inputs = {k: v.to(model.device) for k, v in inputs.items()} |
| |
| outputs = model.generate(**inputs, max_new_tokens=256) |
| text = processor.decode(outputs[0], skip_special_tokens=True) |
| print(text) |
| ``` |
|
|
| ## Training Details |
|
|
| - **Training Data**: 10,000 synthetic Polish document images |
| - **Categories**: Addresses, invoice lines, receipt lines, dates, names, prices, phrases |
| - **Hardware**: Trained with LoRA to enable fine-tuning on consumer hardware (4-6GB VRAM) |
| - **Epochs**: 1 epoch over full dataset |
| - **Optimizer**: AdamW with linear learning rate schedule |
|
|
| ## Baseline Performance (Pre-Fine-Tuning) |
|
|
| Baseline PaddleOCR-VL performance on Polish test set: |
|
|
| | Metric | Value | |
| |--------|-------| |
| | Character Error Rate (CER) | 5.58% | |
| | Word Error Rate (WER) | 13.37% | |
| | Exact Match | 74.00% | |
| | Diacritic Accuracy | 74.14% | |
|
|
| Improved version: |
| Summary: |
| | | Baseline | Fine-tuned | |
| |-------|----------|------------| |
| | CER | 5.58% | 1.60% | |
| | WER | 13.37% | 7.21% | |
| | Exact | 74% | 76% | |
|
|
| Key diacritic confusions in baseline: |
| - `ł` frequently confused with `l` or `t` |
| - `ę` sometimes rendered as `e` |
| - `ś` confused with `š` |
|
|
| ## Limitations |
|
|
| - Optimized for printed Polish text; handwritten recognition may vary |
| - Best results on clean document scans; heavily degraded images may still have errors |
| - Inference requires loading both base model and LoRA weights |
|
|
| ## License |
|
|
| Apache 2.0 (same as base model) |
|
|
| ## Citation |
|
|
| If you use this model, please cite: |
|
|
| ```bibtex |
| @misc{rysocr2024, |
| title={RysOCR: Polish OCR LoRA for PaddleOCR-VL}, |
| author={Kacper Wikieł}, |
| year={2024}, |
| publisher={Hugging Face}, |
| url={https://huggingface.co/anon13370/RysOCR} |
| } |
| ``` |
|
|
