Use ansible to distribute large files in conjunction with peer-software

A description of the application scenario

Now I need to distribute Logstash new version of RPM package to 50+ number of servers, about 220MB, directly using ansible Copy command, the command is as follows:

Ansible all-m copy-a "src=/opt/software/logstash/logstash-agent-2.3.3-fb.centos6.x86_64.rpm dest=/opt/software/ Logstash

In the process of implementation, soon received Zabbix network monitoring alarm, alarm project is instantaneous traffic change more than 5Mbps. At the same time, some servers are executed quickly, there are many SSH connection errors, Ansible card dead.

The pipelining feature that turns on Ansible is still stuck on the SSH connection. Distribution file failed!!

Therefore, it is very fast to use ansible to distribute execution commands, but it is a big problem to use ansible to handle a slightly larger file distribution, even if it takes a little time to endure, but the bandwidth of a single distribution point directly affects the efficiency of distribution.

For large file distribution, the first thing that comes to mind is BitTorrent, which enables fast distribution, saving bandwidth and improving efficiency.

Two-PEER software introduction

Here we use Twitter's open source murder. Twitter uses it to distribute large files to complete code updates. In the early days, Twitter had a headache for distributing code to tens of thousands of servers every day, and it was a big bottleneck to distribute code from a central code server to thousands of other nodes, since the execution time of the distribution code was linearly related to the nodes that needed to update the code, and the more nodes, the longer the distribution time. To solve this problem, Twitter has abandoned its previous centralized architecture and turned to a distributed architecture called murder. After using murder, they previously needed a 40-60-minute code release task that could now be completed in less than 12 seconds.

650) this.width=650; "Src=" http://blogs.cornell.edu/info4220/files/2013/04/     Screen-shot-2013-04-05-at-2.05.01-pm-10o2wu4-300x224.png "alt=" screen-shot-2013-04-05-at-2.05.01-pm-10o "/> 650) this.width=650; "Src=" http://blogs.cornell.edu/info4220/files/2013/04/ Screen-shot-2013-04-05-at-2.06.03-pm-2f8q48m-300x201.png "alt=" Screen-shot-2013-04-05-at-2.06.03-pm-2f8 "/>

650) this.width=650; "src=" Http://blogs.cornell.edu/info4220/files/2013/04/images-scm7r0.jpeg "alt=" Images-scm7r0.jpeg "/>

Three use ansible to perform distribution commands

Reference Documentation:

http://blogs.cornell.edu/info4220/2013/04/05/murder-distributed-large-scale-code-deployment/

http://www.royans.net/wp/tag/tools/

This article is from the Linux SA John blog, so be sure to keep this source http://john88wang.blog.51cto.com/2165294/1793080

Use ansible to distribute large files in conjunction with peer-software