Full FIQA Q8_0 evaluation
This report covers all 648 FIQA test queries, reranking the frozen top-100 retriever candidates for each query (64,800 scored pairs). All table rows use the same deterministic ranking policy.
| Model | NDCG@10 | MAP@10 | Recall@10 | MRR@10 | NDCG@100 |
|---|---|---|---|---|---|
| Retriever | 0.46985 | 0.38419 | 0.55188 | 0.54942 | 0.53769 |
| Transformers BF16 | 0.62486 | 0.54508 | 0.68604 | 0.70187 | 0.65898 |
| GGUF Q8_0 | 0.62444 | 0.54426 | 0.68654 | 0.70146 | 0.65835 |
| Check | Result |
|---|---|
| Q8_0 NDCG@10 drop vs BF16 | 0.00042 |
| Q8_0 MRR@10 drop vs BF16 | 0.00041 |
| Top-1 agreement with BF16 | 0.956790 |
| Mean top-10 set overlap | 0.961728 |
| Pairwise order consistency | 0.985426 |
| End-to-end throughput | 11.787 pairs/s |
| Completed shards | 81 |
| Full FIQA quality gate | accepted |
The Q8_0 release passes when both NDCG@10 and MRR@10 drops relative to Transformers BF16 are no greater than 0.005. This model passed both checks.
All 64,800 full-evaluation scores are finite, and all 81 shard states completed successfully.