FourierPET: Frequency-Selective ADMM Unrolling for Low-Count PET Reconstruction with Better Fidelity at 0.44M Parameters

Executive Summary

TL;DR

Low-count PET reconstruction suffers from multiple overlapping failure modes: stochastic Poisson noise, photon starvation, and attenuation correction bias. FourierPET argues that these artifacts are hard to separate in image space but become more structured in Fourier space: high-frequency phase is mostly damaged by low-count noise, while low-frequency amplitude is mainly suppressed by attenuation-related bias. Based on this observation, the paper builds an ADMM-unrolled network that treats these components differently, yielding state-of-the-art PSNR/SSIM on three PET datasets with only 0.44M parameters.

Background Positioning

This is not merely another leaderboard-oriented PET reconstructor. Its more important contribution is conceptual: it proposes a physically grounded spectral factorization of low-count PET degradation and then bakes that factorization into the optimization architecture. In that sense, the paper sits between a SOTA engineering system and a modeling paper that upgrades the inductive bias of deep unrolling.

Problem & Motivation

Low-count PET is a classic inverse problem made worse by three distinct corruption sources:

• Poisson noise from reduced photon statistics
• Photon starvation that removes fine structural detail
• Attenuation correction (AC) errors that create systematic intensity bias

The issue is not only that these are severe, but that they are mixed together in image space. A network trained with a generic pixel loss sees one corrupted image and is asked to fix everything at once. That works to a point, but it encourages averaged correction behavior:

• denoise broadly,
• sharpen somewhat,
• recover contrast heuristically,
• but without understanding which artifact lives where.

The authors’ central claim is that this entanglement is partly an artifact of representation. In Fourier space, the degradations become more separable:

• High-frequency phase perturbation is strongly associated with Poisson noise and photon scarcity.
• Low-frequency amplitude suppression is strongly associated with AC bias.

That is the paper’s real intellectual move. Once this spectral asymmetry is identified, the reconstruction objective no longer needs to treat all frequencies and all complex-spectrum components equally.

The supporting preliminary study is stronger than a typical motivation section. The paper uses:

• amplitude-phase swap experiments between full-count and low-count scans,
• quantitative PSNR/RMSE/SUVmax comparisons, and
• frequency deviation profiling using radial bands and DWT subbands.

The takeaway is intuitive and important:

• If you replace the phase with that from a better scan, structural sharpness improves.
• If you replace the amplitude, global contrast and intensity recover.
• Therefore, the correction target is not just “remove noise,” but “repair the right spectral variable at the right band.”

This is why a unified image-domain penalty is suboptimal: it ignores the geometry of the degradation mechanism.

Methodology - The Core

From Regularized Reconstruction to Deep Unrolling

The paper begins with the usual inverse formulation: [ x^* = \arg\min_x \frac{1}{2}|y-\mathbf{A}x|_2^2 + \lambda g(x), ] where:

• (x) is the PET image,
• (y) is the measured sinogram,
• (\mathbf{A}) is the PET system matrix,
• (g(x)) is the regularizer.

Using ADMM, the problem is split into alternating updates over:

• the reconstruction variable (x),
• an auxiliary variable (z),
• and a dual variable (u).

This leads to a natural unrolled architecture, where each iteration becomes one stage of the network.

What matters is how the authors reinterpret the three ADMM subproblems:

1. x-update: enforce measurement consistency and global spectral coherence
2. z-update: apply frequency-aware prior correction
3. u-update: stabilize convergence with an adaptive dual step

This preserves the optimization skeleton while inserting neural modules only where they are useful.

1. SCM: Spectral Consistency Module

The exact ADMM x-update requires solving: [ x^{k+1} = (\mathbf{A}^ op \mathbf{A} + \rho I)^{-1}(\mathbf{A}^ op y + \rho(z^k-u^k)), ] which is expensive and not naturally aware of learned structure.

SCM acts as a learnable surrogate for this inverse operator. Its design is hybrid:

• Physics-constrained part: it uses (\mathbf{A}^ op), so the update remains tied to the sinogram.
• Spatial prior part: parallel depthwise convolutions with 3x3 and 5x5 kernels capture local metabolic structures.
• Global spectral part: SSFNO blocks process FFT features through a State-Space Duality mechanism to model long-range dependencies.

The phrase “spectral consistency” is well chosen. This module is not just denoising in Fourier space; it tries to keep the reconstruction globally coherent while still respecting the forward model.

Why should this help?

• PET artifacts are not purely local.
• A lesion boundary may be locally visible but globally inconsistent with the measured sinogram.
• Conversely, pure physics inversion without a learned prior is brittle under extreme count reduction.

SCM sits in that middle ground: a learned inverse step with explicit measurement anchoring.

The ablation results show this module is not superficial. Removing the (\mathbf{A}^ op)-based constraint causes a collapse to SSIM 0.8328 / PSNR 22.55 / RMSE 0.0849, far worse than the full SCM at 0.9740 / 35.19 / 0.0188. That is a strong signal that the paper’s gains do not come from generic neural capacity alone; the physics prior is doing real work.

2. APCM: Amplitude-Phase Correction Module

This is the most distinctive part of the paper.

The z-update introduces a spectral regularizer: [ g(z)=\lambda_a \mathcal{R}{amp}(|\mathcal{F}(z)|)+\lambda_p \mathcal{R}{phase}(\angle \mathcal{F}(z)). ]

The exact proximal operator is not tractable, so the authors build APCM as a learnable one-step approximation under two assumptions:

• Band-wise near separability
• Per-frequency amplitude-phase decoupling

This is an approximation, and the paper is honest that it lacks a full formal derivation. But as a modeling move, it is highly sensible.

APCM works in three stages:

Spectral sharding

A single-level Haar DWT splits the image into four subbands:

• LL
• HL
• LH
• HH

Within each subband, local FFT is applied to obtain amplitude and phase.

This matters because it localizes the correction target:

• low-frequency behavior is mostly in LL
• high-frequency instability is concentrated in HH

Directional correction

Two separate branches are used:

• Amplitude branch: focuses especially on correcting the LL amplitude to recover low-frequency suppression caused by AC bias.
• Phase branch: stabilizes high-frequency phase, especially in HH, to recover structure and suppress noise.

This is the paper’s main operationalization of its scientific hypothesis. Instead of a monolithic “frequency module,” it asks:

• which spectral variable is wrong,
• in which band,
• for which physical reason?

Spectral fusion

The corrected amplitude and phase are recombined into complex spectra, then transformed back via inverse FFT and inverse DWT.

The best part of APCM is that it avoids the usual spectral-restoration pitfall: “if Fourier space is useful, correct everything everywhere.” The appendix explicitly compares targeted correction against full-band correction. The targeted version gets slightly lower RMSE tradeoff but higher structural fidelity:

• Targeted correction: SSIM 0.9740, PSNR 35.19, RMSE 0.0190
• Full-band correction: SSIM 0.9709, PSNR 35.22, RMSE 0.0187

That is revealing. Full-band correction improves global fit a bit, but harms structure. In other words, indiscriminate spectral editing overwrites clean regions. The paper’s selective strategy seems to be the right bias.

3. DAM: Dual Adjustment Module

Standard ADMM updates the dual variable with a fixed step size. In ill-posed low-count PET, that is fragile:

• too small: convergence is slow
• too large: updates oscillate

DAM turns the dual step size (\mu) into a learnable scalar: [ u^{k+1}=u^k + \mu(x^{k+1}-z^{k+1}). ]

This is a relatively lightweight idea, but a good one. Deep unrolling often inherits optimization parameters from classical solvers without asking whether they should remain fixed after the solver becomes learnable. DAM says no: once the network is data-adaptive, the dual ascent strength should be adaptive too.

The appendix shows DAM reduces primal residual faster and more stably. This is less flashy than the spectral branch, but important for training and stage-to-stage behavior.

Experiments & Results

Datasets and setup

The evaluation spans three datasets:

• BrainWeb: simulated brain PET, 20% and 40% count
• In-House: 60 pediatric whole-body PET scans, 1% and 10% count
• UDPET: 206 brain scans at dose reduction factor 100

The diversity is meaningful here. PET reconstruction papers often risk overfitting to one anatomy or scanner style. Testing across simulation, clinical in-house data, and public low-dose data strengthens the paper’s claim.

SOTA comparison

The main comparison table is strong.

On BrainWeb (20% count):

• FourierPET: SSIM 0.9859, PSNR 35.36, RMSE 0.0198
• RED: 34.45 PSNR
• LCPR-Net: 33.75 PSNR
• AutoContextCNN: 33.64 PSNR

On In-House (1% count):

• FourierPET: SSIM 0.9740, PSNR 35.19, RMSE 0.0188
• LCPR-Net: 34.95 / 0.0206
• CNNBPnet: 34.62 / 0.0200
• RED: 34.15 / 0.0192

On UDPET DRF-100 (1% count):

• FourierPET: SSIM 0.9083, PSNR 27.98, RMSE 0.0437
• LCPR-Net: 27.77 / 0.0446
• FBPnet: 27.36 / 0.0463
• RED: 26.51 / 0.0474

What is especially notable is parameter efficiency:

• FourierPET: 0.44M
• DGLM_u: 0.68M
• FBPnet: 21.35M
• RED: 28.93M
• LCPR-Net: 75.93M

So this is not one of those “wins by being huge” systems. It wins while being compact.

Qualitative behavior

The qualitative figures support the quantitative gains. FourierPET preserves:

• lesion contrast,
• edge sharpness,
• global radiotracer distribution,
• while avoiding the over-smoothing seen in some baselines.

It is particularly convincing that the paper also visualizes Fourier-domain log-magnitude errors, not just image-space slices. Since the method’s claim is spectral, showing reduced spectral distortion is the correct form of evidence.

Ablation study

The ablation results are well aligned with the claimed design logic.

Core modules

Starting from a plain unrolled ADMM baseline:

• baseline: SSIM 0.940 / PSNR 33.15
• +SCM: 0.971 / 34.62
• +APCM: 0.967 / 34.05
• +SCM + APCM: 0.974 / 35.19

Interpretation:

• SCM contributes more overall, likely because measurement-consistent inversion is foundational.
• APCM still adds a real gain, especially when combined with SCM.
• The combination is additive rather than redundant.

SCM submodules

Removing any part hurts performance:

• without (\mathbf{A}^ op): catastrophic degradation
• without SSFNO: worse global spectral modeling
• without spatial module: modest but clear drop

This supports the idea that local and global priors are complementary.

APCM branches

• phase-only correction boosts structural fidelity
• amplitude-only correction improves global bias metrics
• combining both gives the best overall balance

That outcome is exactly what the paper’s hypothesis predicts, which is reassuring. The method is not merely working empirically; its behavior matches the intended decomposition.

Efficiency trade-offs

The appendix reports that larger unroll depth (K) or module depth (N) improves accuracy further, but with higher inference cost.

For example:

• K=3: 0.44M params, 20.25 s, SSIM 0.9740
• K=10: 1.46M params, 58.87 s, SSIM 0.9790

This suggests the chosen default is a pragmatic operating point rather than the absolute ceiling.

Zero-shot generalization

A pleasant surprise is the reported zero-shot transfer from human PET to in vivo mouse scans. The paper does not overclaim here, but the result is conceptually consistent with the method: if the learned prior captures corruption structure at the spectral level, it may transfer better than an anatomy-specific image prior.

That said, this result deserves broader validation before it is taken as a strong domain generalization claim.

Critical Analysis & Conclusion

What the paper really contributes

The strongest contribution of FourierPET is not “using Fourier transforms in PET.” That idea alone would be too shallow. The real contribution is a structured mismatch model:

• low-frequency amplitude is where AC bias mainly lives,
• high-frequency phase is where count-related corruption mainly lives,
• therefore the prior should be asymmetric across both frequency band and complex-spectrum component.

That is a better inductive bias than generic image-space regression, and the experiments suggest it is also a more parameter-efficient one.

Why this works, in intuitive terms

A useful way to think about the method is:

• Amplitude mostly controls how much energy sits in coarse structures and global intensity fields.
• Phase mostly controls where structures are and how edges align.
• Low-count PET corruption damages these two in different ways.
• FourierPET avoids forcing one network branch to solve both problems with the same representation.

This kind of decomposition often matters more than raw model size.

Limitations

The paper is strong, but not beyond critique.

• The APCM derivation is heuristic.
  The assumptions of band-wise separability and amplitude-phase decoupling are plausible, but not formally justified. This is acknowledged in the appendix and remains the method’s main theoretical weakness.

• Clinical relevance is still indirect.
  Metrics like PSNR, SSIM, RMSE, and even SUV error are helpful, but they are not equivalent to lesion detectability or diagnostic confidence.

• Fixed spectral transforms may be restrictive.
  Using FFT and Haar DWT imposes a hand-designed basis. Different anatomies, tracers, or scanner protocols may benefit from adaptive spectral decompositions.

• Ground truth dependence.
  Training uses full-count OSEM reconstructions as reference, which are themselves algorithm-dependent proxies rather than absolute truth.

Future work

Several follow-up directions are obvious and interesting:

• derive APCM from a more principled variational proximal approximation,
• replace fixed FFT/DWT with learnable or adaptive spectral bases,
• extend the framework to 3D volumetric reconstruction directly,
• evaluate radiologist-facing endpoints such as lesion conspicuity and clinical confidence,
• test whether similar amplitude-phase asymmetry appears in CT, MRI, SPECT, or even non-medical inverse imaging.

Final takeaway

FourierPET is a well-designed deep unrolling paper with a genuinely useful modeling insight. It shows that in low-count PET, the right question is not just how to regularize reconstruction more strongly, but how to regularize the right spectral component at the right scale. That shift from generic restoration to physically aligned spectral correction is what makes the paper stand out.