Highly scalable sparse matrix multiplication

Highly scalable sparse matrix multiplication

Wuzhiguang Mao Chen Han Lei Lijun

Matrix multiplication is a very important basic operation in linear algebra and graph algorithm, while the matrices in large-scale data processing are often sparse matrices. The MapReduce programming framework can effectively support the distributed computing of massive data. Therefore, how to use MapReduce programming framework to realize the multiplication of super large sparse matrix is studied. The traditional matrix multiplication parallel algorithm does not specifically optimize the sparse matrix, resulting in a large amount of unnecessary communication overhead during the calculation. A new algorithm--CRM (column row multiplication) is proposed and compared with the traditional matrix chunking algorithm.

Highly scalable sparse matrix multiplication