szymonrucinski/quand-resnet18-4bit-cifar100

QUAND ResNet18 4-bit - CIFAR100

A ResNet18 model trained with QUAND (Quantization-Aware Noise-injection Training) for 4-bit integer-weight deployment.

Model Details

Property	Value
Architecture	ResNet18 (torchvision)
Quantization	4-bit (16 levels, w_range=[-8, 7])
Dataset	CIFAR100
Pretrained init	ImageNet
Training method	QUAND noise annealing
Hypothesis	h058

Performance

Metric	Value
Quantized test accuracy	68.92%
Soft test accuracy	68.92%
Mean abs distance to integer	1.72e-06
Quantization MSE	9.11e-12

Training Configuration

Parameter	Value
Noise scales	12
Epochs per scale	120
Learning rate	0.005
Optimizer	Adam
Cosine LR	True
Alpha ramp	0.0 -> 1.0
Snap rate	0.0 -> 0.01

W&B Run

Training logs and metrics: https://wandb.ai/szymonindy/quand/runs/dsroqn83

How QUAND Works

QUAND injects shaped noise (triangular PDF) during training, progressively annealing its amplitude to zero. Combined with an alpha ramp that attracts weights toward integer values, this trains networks whose weights naturally converge to integers -- no post-training quantization needed.

Usage

import torch
state_dict = torch.load("model.pt", map_location="cpu")
# Weights are already integer-valued (within [-8, 7])
# Load into a standard torchvision ResNet18
from torchvision.models import resnet18
model = resnet18(num_classes=100)
model.load_state_dict(state_dict)

Citation

@software{quand2026,
  title={QUAND: Quantization-Aware Noise-injection Training},
  author={Rucinski, Szymon},
  year={2026},
  url={https://github.com/szymonrucinski/qand}
}