Bachelor's thesis presentation. Kevin is advised by Manuel Geiger.
Previous talks at the SCCS Colloquium
Kevin Zheng: Sparse Tensor Contraction: A Comparative Benchmarking Study
SCCS Colloquium |
Tensor contraction is a central operation in scientific computing, but the computational and memory costs grow rapidly with tensor order and dimensionality. While dense libraries such as Eigen provide robust baselines, they become inefficient when tensors are extremely sparse, since zeros are redundantly stored and processed. This thesis evaluates three sparse tensor formats Coordinate Format (COO), Compressed Sparse Fiber Format ( CSF), and Hierarchical Coordinate Format (HiCOO ) against the dense baseline provided by the Eigen library. We benchmark contraction performance (GFLOPS ) and peak memory usage across varying dimensions and sparsity levels. Results show that sparse formats outperform dense methods only at high sparsity levels, while Eigen remains superior at lower and medium sparsity. With extreme sparsity, sparse tensor formats’ memory usage scales favorably, but due to indexing overhead, it grows considerably as non-zero density increases. Arithmetic Intensity estimates further suggest that our sparse contractions are generally compute-bound for higher sparsity. Overall, the sparse formats offer clear advantages in highly sparse settings, but dense methods remain preferable in many practical cases.