First-hand information for everyone
This paper proposes a simple, geometric way to compare two datasets that live in the same feature space. The authors treat the question “can the same vector z be written both as Ax and as By?” as a primitive comparison. Here A and B are data matrices whose columns are individual samples, and the co-span constraint Ax = By = z captures whether z can be represented from A and from B in the shared ambient space. From this view they use a matrix tool to give a per-sample diagnostic called the alignment angle θ(z), which runs from 0 to π/2. Values near 0 mean “explained more by A”, values near π/2 mean “explained more by B”, and values near π/4 indicate comparable, shared explanation.
The technical backbone is the generalized singular value decomposition (GSVD). In one common GSVD form the authors write A = H C U and B = H S V, with the diagonal-like factors C and S satisfying C^T C + S^T S = I. Intuitively, H sets a shared coordinate frame in the ambient space, while the entries of C and S expose which directions are primarily supported by A, by B, or by both. For a given sample z they compute its coordinates c(z) = H^† z (using the Moore–Penrose pseudoinverse H^†) and then form GSVD-weighted costs a(z) = ||C^† c(z)||_2 and b(z) = ||S^† c(z)||_2. The alignment angle is then θ(z) = arctan(a(z)/b(z)). The same angle can be seen directly from minimal-norm coefficient vectors x and y satisfying Ax = By = z, where θ(z) = arctan(||x||_2 / ||y||_2).
The authors illustrate the score on the MNIST image dataset. They report angle distributions and examine representative GSVD directions that make the shared and dataset-specific geometry explicit. They also show a simple binary classifier built from θ(z) as an example of how the angle can be used as an interpretable diagnostic signal. The main role of θ(z) in the paper is as a per-sample geometric diagnostic, not as a claim of state-of-the-art classification performance.
Why this matters: many existing dataset-comparison tools return a single similarity number or rely on trained models and assumed correspondences. The GSVD-based angle gives a complementary, geometry-grounded view. It makes shared versus dataset-specific directions explicit, and it yields a per-sample score that can flag examples that are more compatible with one dataset than the other. That can be useful for diagnostics, filtering, or prioritizing instances in transfer or auditing scenarios.
Important caveats and uncertainty. The method is linear: it compares datasets through linear relations and a common ambient basis. A sample z must lie in the intersection of the column spaces of A and B (col(A) ∩ col(B)) for the co-span relation to be defined; otherwise the sample is “non-relational” under this framework. The computations use matrix pseudoinverses (optionally truncated for numerical stability), so numerical choices matter in practice. The paper illustrates the idea on MNIST and provides derivations and GSVD properties in appendices, but extensive validation across many kinds of real-world data is not presented in the excerpt. In short, GSVD and the alignment angle offer an interpretable linear diagnostic, but they are limited to settings where linear structure in a shared feature space is a reasonable model.