First-hand information for everyone
This paper explains when and why a broad class of iterative algorithms behaves the same on many large matrices, including some deterministic transform matrices. The algorithms studied are general first‑order methods (GFOM): they alternate multiplying a state vector by a matrix and applying simple scalar nonlinearities entrywise. A well‑known special case, approximate message passing (AMP), often has a particularly simple limit: the state at each step looks Gaussian in large dimensions. Until now, that Gaussian picture was mainly justified for very random matrices and remained mysterious for more structured, deterministic inputs.
The authors attack the problem with a combinatorial and diagrammatic approach based on the matrix’s limiting “traffic distribution.” Informally, the traffic distribution records the limiting values of all permutation‑invariant polynomials in the matrix entries as the matrix size grows. Using these diagram expansions, they compute the traffic distribution for the first nontrivial deterministic examples, including minor variants of the Walsh–Hadamard transform and the discrete sine and cosine transforms. From that calculation they derive the asymptotic dynamics of GFOM on those matrices, resolving parts of conjectures made by Marinari, Parisi, and Ritort (1994).
They also design a new AMP‑type iteration that unifies several previous AMP variants and extends to new input types. For a large and natural class of traffic distributions — a class that includes both many random ensembles and the deterministic transforms they studied — the algorithm’s limiting dynamics remain Gaussian, but in a conditional sense: the state is asymptotically Gaussian given some latent random variables. The paper gives a simple combinatorial interpretation of the so‑called Onsager correction term that appears in AMP analyses, answering questions posed recently by Wang, Zhong, and Fan (2022). The authors describe this new method as a “treelike AMP” in their development.
Why this matters: knowing the limiting, large‑size behavior of these iterative methods helps explain why AMP‑style algorithms can work well in practice, even on structured deterministic matrices. A concrete description of the limits also helps researchers design or tune algorithms to achieve better performance in optimization and inference tasks where the input is large and has some fixed structure.
Important caveats: the results are asymptotic statements that hold as the matrix dimension n grows large. The Gaussian description is conditional on latent variables in the more general inputs, so it is not always an unconditional Gaussian law. The analysis depends on membership in the class of traffic distributions the paper treats, so the conclusions do not automatically apply to every possible structured matrix. Finally, the techniques are combinatorial and diagrammatic; they give precise answers for the families the paper covers but do not claim a complete classification of all matrices or algorithms.