PlumaDINO-vits16plus-preview

PlumaDINO-vits16plus-preview is an independently fine-tuned image classifier for living bird species, built on the distilled DINOv3 ViT-S+/16 backbone. This preview supports aspect-ratio-preserving inputs and loads directly with Transformers.

Model weights: non-commercial use only. See the model weights license.

Quick start

Install the dependencies in requirements.txt in your own environment. Run this example from the parent of the downloaded model folder. It uses CPU; set device = "cuda" to use a GPU. After publication, a Hub repository ID may replace the local folder path.

import torch
from PIL import Image
from transformers import AutoImageProcessor, AutoModelForImageClassification

model_id = "./PlumaDINO-vits16plus-preview"
device = "cpu"
torch.set_float32_matmul_precision("highest")
torch.backends.cuda.matmul.allow_tf32 = False
torch.backends.cudnn.allow_tf32 = False
processor = AutoImageProcessor.from_pretrained(
    model_id, trust_remote_code=True, use_fast=False
)
model = AutoModelForImageClassification.from_pretrained(
    model_id, trust_remote_code=True, dtype=torch.float32
).to(device).eval()

with Image.open("bird.jpg") as image:
    inputs = processor(images=image, return_tensors="pt").to(device)
with torch.inference_mode():
    logits = model(**inputs).logits[0]
scores = logits.softmax(-1)
for index in torch.argsort(logits, descending=True, stable=True)[:5].tolist():
    print(model.config.id2label[index], f"{scores[index].item():.2%}")

For an offline local copy, pass local_files_only=True to both loaders. AutoModel.from_pretrained(..., trust_remote_code=True) also loads the classifier. The folder includes the custom inference code; a separate DINOv3 checkout or base-weight download is not needed. example_inference.py is an additional command-line entry point.

Inference precision

FP32, FP16, and BF16 use the usual Transformers loading and PyTorch conversion APIs. For example, to run FP16 on a GPU:

model = AutoModelForImageClassification.from_pretrained(
    model_id, trust_remote_code=True, dtype=torch.float16
).to("cuda").eval()
with Image.open("bird.jpg") as image:
    inputs = processor(images=image, return_tensors="pt").to(
        model.device, dtype=model.dtype
    )
with torch.inference_mode():
    logits = model(**inputs).logits

Use dtype=torch.bfloat16 for BF16 or dtype=torch.float32 for FP32. model.half(), model.bfloat16(), and model.to(dtype=...) are also supported. The processor's .to(...) handles both regular batches and different-size image lists. Existing external autocast usage remains supported. FP8 and quantized weight loading are unsupported.

Reduced precision can change logits and individual predictions; numerical closeness to FP32 does not establish higher recognition accuracy. The benchmark table below uses the original FP32 protocol with TF32 disabled. Loading and inference APIs are checked with Transformers 4.56.1 and 5.19.0.

Model and inputs

• Backbone: distilled DINOv3 ViT-S+/16, with separate Q/V LoRA adapters.
• Classifier: concatenated CLS and mean patch features, followed by a cosine classifier. Inference uses raw masked logits without training-time logit adjustment.
• Labels: standardized AviList species. labels.json provides stable indices, scientific and English names, family and support status. Unsupported slots retain their indices and receive zero probability.
• Anyres preprocessing: EXIF orientation, RGB conversion, aspect-preserving resize when needed, black padding to patch boundaries, and ImageNet normalization. The default canvas budget is one megapixel. Images are not stretched into squares.
• Batching: pass a list of images to the processor. Different shapes retain their own canvases; matching shapes are batched without padding all images to a common rectangle. Keep batch composition fixed for numerical comparisons.

Scores are uncalibrated closed-set probabilities. This preview does not provide an unknown-species or non-bird detector, and rare species remain challenging.

Data

Fine-tuning uses a curated selection from LaSBiRD, BioTrove and TreeOfLife, with standardized species labels, duplicate filtering and held-out identity exclusion. The collected-data test split was frozen separately from the fine-tuning selection. Training images are not distributed with this model.

Benchmarks

All values below belong to the source checkpoint packaged in this preview and are percentages. Evaluation uses deterministic aspect-preserving inputs, FP32 with TF32 disabled, and ranking over the complete supported label space. Candidate species are not restricted to those present in each benchmark.

Evaluation set	Top-1	Top-5	Macro Top-1	Macro Top-5	Macro F1
NABirds	72.82	92.47	74.22	93.14	80.29
Rare Species (birds)	75.11	86.49	75.12	86.49	83.09
Held-out collected birds	80.23	91.68	77.59	90.09	76.03

Evaluation scope

• NABirds: the evaluated distribution's test split, mapped from visual labels such as age, sex and plumage variants to standardized species. Unresolved historical species splits are excluded from accuracy. This is a species-mapped evaluation; the original visual-label protocol differs. Split membership comes from the distribution and was not independently matched image by image to the original Cornell split.
• Rare Species: only the bird subset (Aves). The distribution calls its split train; these images are used here exclusively for evaluation.
• Held-out collected birds: the frozen held-out selection, excluded from fine-tuning through the project's identity and duplicate controls.
• Metrics: Top-1/Top-5 are image-weighted; macro scores give equal weight to each evaluated species. Macro F1 uses prediction counts over the complete evaluated set. No location prior or true-label distribution mask is applied.
• Overlap: exact file and decoded-pixel checks found no matches between the public benchmark images and the fine-tuning selection. Near-duplicate overlap and overlap with backbone pretraining data have not been ruled out.

These are audited results for the selected source checkpoint. Export validation separately compares native inference, portable Transformers loading, and a save/load roundtrip on deterministic engineering inputs; it is not a new full benchmark run.

Authorship, attribution, and license

This release: bird-specific fine-tuning, data curation, evaluation, and packaging by the independent publisher of this model.

Upstream DINOv3: the original backbone and pretrained weights are by Meta Platforms, Inc. and affiliates. native_bird_backbone.py adapts inference operations from the DINOv3 source.

Taxonomy: AviList.

Model weights license

PlumaDINO publisher terms — non-commercial use only. Use, modification, and redistribution of the PlumaDINO-specific weights and modifications in this release are permitted for non-commercial purposes only. Commercial use of these weights or their derivatives, including use in commercial products or services, is prohibited. Redistribution must retain these terms and the DINOv3 License.

Upstream license: LICENSE.upstream.md contains the original DINOv3 agreement from Meta. Redistribution of the incorporated DINOv3 materials and their derivatives remains subject to that agreement. Meta is the upstream licensor; this bird classifier is independently fine-tuned and published.