CarolinePascal commited on
Commit
3f8d017
Β·
unverified Β·
1 Parent(s): f6b9039

docs: add CPU-only note to hero and uniformize polarity phrasing

Browse files

Mention measurements are CPU-only in the shared page hero (RGB + Depth),
and standardize About-page metric descriptions from "Lower/Higher = …"
to "Lower/Higher is …" across both benchmarks.

Files changed (2) hide show
  1. app.py +2 -1
  2. src/schema.py +7 -7
app.py CHANGED
@@ -528,7 +528,8 @@ def _build_hero(bench: schema.BenchmarkConfig, total_rows: int, repo_count: int)
528
  gr.Markdown(
529
  "**Open benchmark**\n\n"
530
  f"# {bench.title}\n\n"
531
- f"{bench.subtitle}",
 
532
  elem_classes="hero-copy",
533
  )
534
  stats = gr.Markdown(
 
528
  gr.Markdown(
529
  "**Open benchmark**\n\n"
530
  f"# {bench.title}\n\n"
531
+ f"{bench.subtitle}\n\n"
532
+ "_All measurements are currently CPU-only._",
533
  elem_classes="hero-copy",
534
  )
535
  stats = gr.Markdown(
src/schema.py CHANGED
@@ -331,9 +331,9 @@ A benchmark for video encoding and decoding in the context of robotics datasets.
331
  </ul>
332
  <p><b>Fidelity</b> β€” every decoded frame is compared against the uncompressed source, pixel-for-pixel.</p>
333
  <ul>
334
- <li><b>PSNR (dB)</b> β€” peak signal-to-noise ratio in dB. Logarithmic, so +3 dB β‰ˆ half the error. 40+ is excellent, 30 is acceptable, 20 is visible artefacts. Higher = closer to the source.</li>
335
- <li><b>SSIM (0–1)</b> β€” structural similarity index, dimensionless. Perceptual β€” weights luminance, contrast, structure. 0.95+ is good, 0.80 is degraded. Higher = structure better preserved.</li>
336
- <li><b>MSE (pxΒ²)</b> β€” mean-squared error in squared 8-bit pixel intensities (0–65025). Lower = less pixel error; 0 = identical.</li>
337
  </ul>
338
  </p>
339
 
@@ -653,10 +653,10 @@ error introduced by quantization and lossy codecs.</p>
653
  </ul>
654
  <p><b>Depth fidelity</b> β€” every decoded frame is de-quantized back to metric depth and compared against the source depth map, pixel-for-pixel; errors are reported in meters. Invalid pixels (zero / no return) are excluded so they don't skew the error.</p>
655
  <ul>
656
- <li><b>RMSE (m)</b> β€” root-mean-squared error between decoded and source depth, in meters. Squares the residuals, so it punishes large per-pixel misses harder than MAE. Lower = decoded depth closer to the source.</li>
657
- <li><b>MAE (m)</b> β€” mean absolute error in meters. The plain average miss per pixel. Lower = smaller average miss.</li>
658
- <li><b>AbsRel</b> β€” mean of <code>|decoded βˆ’ source| / source</code>. A relative error, so a 5&nbsp;cm miss at 1&nbsp;m counts far more than the same miss at 10&nbsp;m; the standard depth-estimation metric. Lower = less error relative to the true depth.</li>
659
- <li><b>Ξ΄&lt;1.25 / 1.25Β² / 1.25Β³</b> β€” accuracy thresholds: the fraction of pixels whose ratio <code>max(d/d*, d*/d)</code> stays under 1.25, 1.25Β² and 1.25Β³. Higher = more pixels within tolerance (1.0 = every pixel within threshold); the cubed threshold is the most forgiving.</li>
660
  <li><b>Quantized variants</b> (<code>Quant …</code> columns, hidden by default) β€” the very same metrics computed after quantization but <i>before</i> the codec. They isolate the error you lose just by squeezing continuous depth into an integer pixel range, so the gap between a metric and its <code>Quant</code> twin is the codec's own contribution.</li>
661
  </ul>
662
  </p>
 
331
  </ul>
332
  <p><b>Fidelity</b> β€” every decoded frame is compared against the uncompressed source, pixel-for-pixel.</p>
333
  <ul>
334
+ <li><b>PSNR (dB)</b> β€” peak signal-to-noise ratio in dB. Logarithmic, so +3 dB β‰ˆ half the error. 40+ is excellent, 30 is acceptable, 20 is visible artefacts. Higher is closer to the source.</li>
335
+ <li><b>SSIM (0–1)</b> β€” structural similarity index, dimensionless. Perceptual β€” weights luminance, contrast, structure. 0.95+ is good, 0.80 is degraded. Higher is structure better preserved.</li>
336
+ <li><b>MSE (pxΒ²)</b> β€” mean-squared error in squared 8-bit pixel intensities (0–65025). Lower is less pixel error; 0 = identical.</li>
337
  </ul>
338
  </p>
339
 
 
653
  </ul>
654
  <p><b>Depth fidelity</b> β€” every decoded frame is de-quantized back to metric depth and compared against the source depth map, pixel-for-pixel; errors are reported in meters. Invalid pixels (zero / no return) are excluded so they don't skew the error.</p>
655
  <ul>
656
+ <li><b>RMSE (m)</b> β€” root-mean-squared error between decoded and source depth, in meters. Squares the residuals, so it punishes large per-pixel misses harder than MAE. Lower is decoded depth closer to the source.</li>
657
+ <li><b>MAE (m)</b> β€” mean absolute error in meters. The plain average miss per pixel. Lower is smaller average miss.</li>
658
+ <li><b>AbsRel</b> β€” mean of <code>|decoded βˆ’ source| / source</code>. A relative error, so a 5&nbsp;cm miss at 1&nbsp;m counts far more than the same miss at 10&nbsp;m; the standard depth-estimation metric. Lower is less error relative to the true depth.</li>
659
+ <li><b>Ξ΄&lt;1.25 / 1.25Β² / 1.25Β³</b> β€” accuracy thresholds: the fraction of pixels whose ratio <code>max(d/d*, d*/d)</code> stays under 1.25, 1.25Β² and 1.25Β³. Higher is more pixels within tolerance (1.0 = every pixel within threshold); the cubed threshold is the most forgiving.</li>
660
  <li><b>Quantized variants</b> (<code>Quant …</code> columns, hidden by default) β€” the very same metrics computed after quantization but <i>before</i> the codec. They isolate the error you lose just by squeezing continuous depth into an integer pixel range, so the gap between a metric and its <code>Quant</code> twin is the codec's own contribution.</li>
661
  </ul>
662
  </p>