2024-11-22 / Qi Yang, Philippe Corboz

LMSVD and Beyond

LMSVD, or limited memory SVD, is a truncated SVD algorithm with practical value in tensor-network calculations. This note records a few ways to make it more useful when the sequence of matrices changes smoothly from one tensor-network iteration to the next.

Basics of LMSVD

LMSVD is built around two ideas. Partially reorthogonalization, derived from PROPACK, keeps orthogonalization costs controlled. LOBPCG applies block optimization to the SVD subproblem, such as finding the leading $k$ eigenvectors of $A^T A$ where $A = U S V^\dagger$.

In tensor networks, the same contraction or truncation problem often appears repeatedly with only small changes. If $A_i = U_i S_i V_i^\dagger$ and $A_{i+1}$ is nearby, previous Krylov information should not be thrown away.

Two acceleration ideas

Krylov space initialization. Initialize the Krylov space through the initY parameter. For nearby matrices, the natural initialization is $Y = V$.
Guard vector recycling. LMSVD uses guard vectors to compute the leading $k$ eigenvectors accurately. Instead of discarding the enlarged $(k + g)$-dimensional space after an iteration, recycle it into the next one.

Benchmarks

Classical 2D Ising model LMSVD benchmark — Classical 2D Ising model benchmark. The minimum iteration count can dominate when the bulk tensor has small bond dimension.

Classical q=5 vector Potts model LMSVD benchmark — Classical q=5 vector Potts model benchmark. The optimized LMSVD path improves performance, though the gain is modest here.

Outlook

The next useful checks are quantum bulk tensors and lower-level LMSVD source changes that reduce the minimum iteration requirement. A randomized one-iteration SVD can be faster, but the accuracy tradeoff needs to be tracked explicitly.