AGF-NO: Restoring the Truncated Band in Spectral Operators
Adaptive Geometry-Aware Fourier Neural Operator — with the complete
controlled-evidence stack, extended depth sweep to 16, a second PDE family,
a deformation baseline, a direct measurement of geometric forgetting, and a
fully programmatic research paper (paper/agfno_paper.pdf).
Core claim: FNO's global mixing channel is structurally band-limited (mode truncation discards everything above the cut). A zero-gated, SDF-derived multiplicative modulation of the spectral weights restores the truncated band by spectral convolution — and the paper measures the whole story: diagnosis (proposition), fix (mechanism), consequence (probe).
1. Attribution result (obstacle Darcy, 3 seeds, identical budgets)
| Variant | rel-L² (global) | rel-L² (ring) | Wall fidelity |
|---|---|---|---|
| FNO baseline | 0.1232 ± 0.0012 | 0.2497 ± 0.0007 | 0.0665 |
| AGF-NO gates-frozen (capacity control) | 0.1197 | 0.2484 | 0.0709 |
| AGF-NO spec-only | 0.1003 | 0.1361 | 0.0069 |
| AGF-NO (full mechanism) | 0.0805 ± 0.0019 | 0.1106 ± 0.0046 | 0.0051 |
| FNO, no penalty | 0.1250 | 0.2577 | 0.0821 |
| AGF-NO, no penalty | 0.0818 | 0.1147 | 0.0091 |
- Capacity control: frozen network (identical graph, gates hard-zeroed) = 0.971× FNO's global error — the gain is NOT extra parameters (5.10M vs 4.81M) or channels (identical 3-channel inputs).
- Mechanism: full AGF-NO beats its own frozen self by −33% global / −56% ring. ~33σ seed separation vs FNO.
- Loss: removing the boundary penalty moves ring error by −3.7%; the advantage is architectural.
2. Depth to 16: collapse vs plateau (the paper's headline)
| depth 4 | depth 6 | depth 8 | depth 16 | |
|---|---|---|---|---|
| FNO rel-L² (global) | 0.1283 | 0.1392 | 0.1444 | 1.0053 (collapse) |
| AGF-NO rel-L² (global) | 0.0813 | 0.1050 | 0.1190 | 0.0846 (depth-stable) |
| FNO probe R² (final block) | 0.567 | 0.475 | 0.502 | 0.0003 (total) |
| AGF-NO probe R² (final block) | 0.626 | 0.607 | 0.652 | 0.633 (plateau) |
At depth 16 the vanilla FNO is worse than predicting the mean, and its latent carries zero linearly decodable geometry — behavioural and representational collapse coincide, as the truncation-band-limitation proposition requires. AGF-NO's probe plateaus within 0.045 R² across depths 4–16 and its ring error improves with depth (0.1163 → 0.1057): depth becomes a resource instead of a liability.
Three-seed replication (all cells re-run, mean ± std):
| depth 4 | depth 8 | depth 16 | |
|---|---|---|---|
| FNO rel-L² | 0.1250 ± 0.0030 | 0.1449 ± 0.0040 | 1.0054 ± 0.0016 |
| AGF-NO rel-L² | 0.0801 ± 0.0010 | 0.1109 ± 0.0058 | 0.0841 ± 0.0012 |
The collapse is essentially deterministic: 11.96× error gap at depth 16 with zero overlap across seeds; FNO's worst-case seed is 1.0036 (still collapsed), AGF-NO's seed spread is 1.4% of its mean. Forgetting at depth is not a tail event — it is what the architecture does. AGF-NO's best cell is depth 4; the honest claim is depth-stable vs depth-collapsing.
3-D replication (the phenomenon is dimension-independent)
The full stack lifts to 3-D obstacle Darcy — spheres and solid tori (multiply-connected, so deformation-based escapes stay structurally unavailable), analytic SDFs, float64 PCG ground truth (every stored field satisfies the discrete PDE to <1e-6 relative residual), zero-gate capacity control unchanged, res 32 → zero-shot 48.
Measured (3 seeds × {fno, agfno} × depth {4, 16}, experiments4/):
| rel-L² (3-D) | depth 4 | depth 16 |
|---|---|---|
| FNO global | 0.144 ± 0.020 | 1.008 ± 0.000 — collapsed, zero seed overlap |
| AGF-NO global | 0.077 ± 0.002 | 0.113 ± 0.003 — depth-stable |
| FNO near-wall | 0.229 ± 0.007 | 0.907 |
| AGF-NO near-wall | 0.085 ± 0.002 | 0.124 (1.45× its depth-4 value) |
The 2-D collapse reproduces unchanged in 3-D: FNO's probe-decodable SDF
information falls 0.494 → 0.020 (R²) from depth 4 to 16 — geometry is
essentially erased from its features — while AGF-NO holds 0.494 at depth 16
(24× more decodable geometry, gate G3). Near-wall error ratio at depth 16:
7.3× (gate G1; FNO's wall violation also explodes 0.034 → 0.771). The
only honest caveat, carried into the paper: AGF-NO's near-wall error does
grow 1.45× from depth 4 to 16 (gate G2 measures stability, not constancy)
— the plateau is not perfectly flat in 3-D, but 0.124 vs FNO's 0.907 is not
a close call. All pre-registered gates pass (ALL_GATES_PASS: true).
3. The forgetting measurement (the novel artifact)
A closed-form linear probe asks, per block: how much SDF information is still linearly decodable from the latent? Shallow sweep (depths 1–4): FNO falls 0.896 → 0.472 monotonically; AGF-NO falls 0.934 → 0.591. The probe ships with unit-tested null (noise → R² ≈ 0) and sanity (linear embeddings → R² > 0.99) controls.
4. Frequency-resolved analysis (P1/P2)
- SDF spectrum: 99.70% of energy below the architectural cut, 0.013% above — the direct tail is tiny; restoration is dominated by multiplicative mixing (measured >20× high-band regeneration; unit-tested, constant modulation = 0).
- P2 confirmed: FNO's high-band ring error 0.0897 → AGF-NO 0.0513 (−43%), high-over-low ratio halved (0.174 → 0.077). The reduction is exactly in the truncated band near walls.
- P1 consistent: wall perturbation influence more wall-confined in FNO (ring/far 1.66) than AGF-NO (1.32).
5. External validity
- Canonical piecewise-constant Darcy (no penalty, no ring loss): FNO 0.1940 / frozen 0.2030 / AGF-NO 0.1450 — ordering replicates with confounds removed. Published references quoted for scale only (FNO 0.0082 @85², Geo-FNO 0.0068; not comparable).
- Deform-FNO baseline: global 0.1058 (beats FNO) but ring 0.2385 (≈ FNO's blindness) — deformation fixes the simply-connected part, cannot touch multiply-connected walls, exactly as predicted.
- Second PDE family (advection–diffusion past fixed-temperature obstacles, zero-shot rollout to 2T): honest near-null — capacity control passes (frozen 1.025× FNO) but mechanism gain is only −2% T / −3.5% ring / −2.5% at 2T. Interpretation in the paper: the mechanism is a targeted fix for wall-anchored difficulty (Darcy's solution is singular at walls; advected thermal layers are smeared downstream). Reported, not hidden.
- A-priori diagnostic (Δρ): predicts where the mechanism helps. Δρ = truncated-band ring-energy fraction of target minus input, on the model's own mode cut. PDE1: +0.0244 (the solve must create high-band wall content) → mechanism gain 0.445. PDE2: −0.0236 (high-band steps are given in the input; diffusion smooths) → gain 0.965 (null). Equal magnitude, opposite sign, matching the observed gains. The naive statistic (target-only ρ) inverts the ranking — documented as a negative result. A new benchmark: compute Δρ before training; it costs two FFTs.
6. Headline run (2,000 samples, 300 epochs, T4)
| Metric | FNO | AGF-NO | Gain |
|---|---|---|---|
| rel-L² (global) | 0.1004 | 0.0541 | −46% |
| rel-L² (ring) | 0.2334 | 0.0627 | −73% |
| Wall fidelity | 0.0769 | 0.0129 | 5.9× |
| Zero-shot 2× super-res | 0.4666 | 0.2295 | −51% |
Repository contents
paper/agfno_paper.pdf— the full 10-page research paper (every number programmatically generated from the JSONs; macros pipeline included).fno_*.pt,agfno_*.pt— headline checkpoints.experiments/— controlled suite: per-variant/per-seed results, ablations, depth sweep, probe (probe/forgetting_probe.json).experiments2/— extended suite: PDE2 matrix + rollout, depth 4/6/8/16, Geo-FNO baseline, extended probe, frequency analysis.experiments3/— 3-seed depth sweep (error bars, replication table).diagnostic/— the a-priori Δρ statistic on both PDE families.experiments4/— 3-D replication: sphere+torus Darcy, depth 4/16, 3 seeds, probe, zero-shot SR to res 48.benchmark/— canonical piecewise-constant Darcy results.results.json,reeval_results.json— headline metrics + local cross-validation (matches remote to 4 decimals).
Reproducibility
One-click public reruns (self-contained kernels, no internet needed): full run · controlled suite · benchmark · extended suite · 3-seed depth sweep · Δρ diagnostic · 3-D replication. Data are byte-deterministic (seed-pinned); 41 unit tests cover the architecture (zero-gate ≡ FNO equivalence, both dimensions), the 2-D and 3-D solvers, SDFs, losses, probe controls, and the band-restoration property.
Honest limitations
Two PDE families, two dimensions (2-D and 3-D synthetic); no external reimplementation at matched settings; probe measures linear decodability (lower bound); PDE2 result shows the mechanism's scope is wall-anchored difficulty; obstacles are static (3-D tori restore multiply-connectedness, but not deforming boundaries over time).