Dmlc: the largest open-source distributed Machine Learning Project

To share and develop code in the distributed machine learning field, the distributed machine learning community (dmlc) has recently been officially released. As an open-source project, dmlc-related code is directly hosted on GitHub and maintained using the apache2.0 protocol. Chen Tianxin (network name), the initiator of the dmlc project, said that the initial idea of the project was to reduce the cost of distributed Machine Learning Development and the time when new algorithms were tested. Currently, this project has integrated machine learning libraries such as xgboost, cxxnet, and Minerva with system components such as RABIT and parameter servers. Next, this article will briefly introduce these basic libraries and components.

Xgboost is a large-scale and distributed general gradient boosting (gbdt, tbrt, or GBM) Library. It implements machine learning algorithms such as gbdt and generalized linear model under the gradient boosting framework. By using distributed computing, xgboost can significantly speed up iteration in algorithms. As a fast and accurate distributed deep learning framework, cxxnet is lightweight, supports multiple GPUs and distributed parallel systems, provides excellent scalability, and provides clear interfaces with other languages. It uses the mshadow library to provide users with a good programming experience and efficient working ability. Unlike cxxnet, Minerva provides an efficient and flexible concurrent deep learning engine. It provides an ndarray programming interface similar to numpy and supports Python and C ++ languages. Its natural concurrency ensures that it can efficiently use multiple GPUs for related computing.

RABIT is a lightweight communication framework that provides Fault-Tolerant interfaces of allreduce and broadcast. It simplifies MPI design and adds Disaster Tolerance support to allreduce and boradcast operations. Its biggest feature is portability, scalability, and reliability. Another communication framework, parameter server, is mainly responsible for providing distributed system support for xgboost and other applications. It supports asynchronous and zero-copy key-value pairs between the worker and the server. The asynchronous parameter server interface and synchronous RABIT interface can basically meet the communication requirements in various distributed machine learning algorithms.

Https://github.com/dmlc

This article is from the "IT technology learning and communication" blog!

Dmlc: the largest open-source distributed Machine Learning Project