A weekly technical update on distributed technology 2015.11.29

Distributed System Practice

1. (Academic paper)Fail at scale reliability in the face of rapid change

http://queue.acm.org/detail.cfm?id=2839461

Important: This article describes the online system stability solution for Facebook. The article covers the configuration management system, hierarchical release, fast rollback, control delay algorithm, concurrency control, prevent avalanche, core service robustness, failure drills and other aspects related to service stability, has great reference value for our system stability (especially the delay control algorithm, which is very effective for solving the long tail problem). Interested students can read further controlling queue delay (http://queue.acm.org/detail.cfm?id=2209336) This paper.

2. Distributed database and inverse paradigm design

http://mp.weixin.qq.com/s?__biz=MzI0MjA1Mjg2Ng==&mid=400528497&idx=1&sn= 6b068d924ba06d030bbb5b147265abfa&scene=0&key= D72a47206eca0ea96172c220b41bf4d2fd453cc51cc34453c93860c47545d81354e638afbef697c33fa0563999baa9bd&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.1+build (15B42) &version= 11020201&pass_ticket=fm72ikhe73ohijfh1jkrnofjhmnntunq8yrmtlceass4plhwdpuno9ncxgwvzjjc

Important: This is an article in some of the author's column technical Articles, which describes how to use database inverse paradigm design to solve distributed database problems in small and medium-sized enterprises.

3. Lambda architecture and recommendations on e-commerce website Practice

https://mp.weixin.qq.com/s?__biz=MzAwMDU1MTE1OQ==&mid=401800864&idx=1&sn= e86e31a4aa6279f5b515f9116da47d59&scene=0&key= D72a47206eca0ea97d58f0d568d7c4baa44b12f09dfe5665b65a347d4ab82b1c013c7fe9eac1339662772de4f51bcc34&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.1+build (15B42) &version= 11020201&pass_ticket=cdp7crtptmbyombyxm9%2b2omffnjuj%2b%2fb%2f%2f7dsgvkouc3uthrqzrli9b6te3shfc6

Key points: This article combines the example of e-commerce, introduces the specific concept of lambda architecture, simply said, lambda architecture is batch processing layer, real-time processing layer and service layer three layer architecture.

4. Kafka file storage mechanism those things

http://mp.weixin.qq.com/s?__biz=MzAwNjQwNzU2NQ==&mid=400594407&idx=3&sn= 463f60dfc80b1849c69ac535c30d6eaf&scene=1&srcid=1125p4pccl6tgbj6tbmi6u5w&key= Ff7411024a07f3ebb56f8edc2777a261d1d46ec1a7056566c26be50f991ed026eafdacc08d0607dc308a0336ed017825&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.1+build (15B42) &version= 11020201&pass_ticket=3zonuafulzfdds9hitoivjtou6dxt7g%2b%2b%2fbgotjoubjnp%2fnvkkjm%2fsgmhn15miqu

Important: The performance of a commercial message queue is good or bad, and its file storage mechanism is designed to measure a Message Queuing service level and one of the most critical indicators. In this paper, from the Kafka file storage mechanism and physical structure, the author analyzes how Kafka achieves efficient file storage and practical application effect.

Service and resource management technology

1. Twitter-based hybrid cloud migration for Docker containers

https://mp.weixin.qq.com/s?__biz=MzAwMDU1MTE1OQ==&mid=401778644&idx=1&sn= 41fc85e861a4ba7aaa4fcea7339f9a08&scene=0&key= D72a47206eca0ea964235f691058d7ff1e6a83cd6091851074d10ff18ee61c48e37ce569010b4e6b0d2d402e9f7e195a&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.1+build (15B42) &version= 11020201&pass_ticket=fm72ikhe73ohijfh1jkrnofjhmnntunq8yrmtlceass4plhwdpuno9ncxgwvzjjc

Important: This paper analyzes the case of micro-blogging using hybrid cloud, explains the advantages of using hybrid cloud and the software stack adopted by Weibo. The practice of microblogging on Docker started relatively early, and the case was very useful.

2. How to use the open VSwitch in Docker

Http://dockone.io/article/837?hmsr=toutiao.io&utm_medium=toutiao.io&utm_source=toutiao.io

Point: We use the computer Room and machine network environment are pre-configured, so you may be a bit unfamiliar with the configuration of the network. Unlike using a physical machine directly, you need to worry about the network configuration of a container or virtual machine, whether it's using Docker or a virtual machine. But unlike physical machines, virtual machines (or Docker containers) are frequently changing, so the network configuration also requires following a change, which has the need for virtual networks, how to make two host different virtual machines over the network communication (also consider overhead as small as possible)? For this SDN came into being, and open vswitch became the most common way of industrial implementation of SDN (for SDN I do not know much, there are wrong places please correct me, also welcome interested students to stand up to share). This article describes how to use the open vswitch in a Docker container to solve the problem of mutual communication between different docker containers.

3. What is Unikernel?

https://mp.weixin.qq.com/s?__biz=MzA5OTAyNzQ2OA==&mid=400591600&idx=1&sn= ad2a950c6a0d23e7e8ff7542fe990c4c&scene=0&key= Ff7411024a07f3eb4614053949f393ad08b523b68bf7df1aedfe47bd1b3cca5816a3b908aaca018f3aa9f98b71c8791a&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.1+build (15B42) &version= 11020201&pass_ticket=9mb1xkc0hqedpjfc7tyecllaequc%2bjr0miadlbozrmw41%2fd9p16mwnwl3kh29aze

Important: If you keep your eye on DevOps Weekly, devops-themed meetings or really interested in technology, you may have heard unikernel many times. Over the past few months, it seems to be getting more and more attention. However, what exactly is Unikernel? Is it what I want? This article has made a brief introduction to this.

Service Scheduling and Trace technology

1. Microblogging in large-scale, high-load system troubleshooting methods

http://mp.weixin.qq.com/s?__biz=MzAwNjQwNzU2NQ==&mid=400579906&idx=3&sn= ec43a81e46e1d59d704dc3654568c9ae&scene=1&srcid=1123jvtjbk7gd2y4jraxsu14&key= D72a47206eca0ea914348206054b50b9c6b48f296779b29b6b4d0a2ccd78de999e420e9784d1d056fd75b65acf41655e&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.1+build (15B42) &version= 11020201&pass_ticket=cdp7crtptmbyombyxm9%2b2omffnjuj%2b%2fb%2f%2f7dsgvkouc3uthrqzrli9b6te3shfc6

Key points: An article that is full of dry goods, from the experience of tracing from a first-line problem. The stability of the team's classmates have a voice, often need to trace the problem of the line, if you are in the usual problem tracing process encountered no clue, may wish to find stability team classmate Chat.

DevOps Technology

1. Atime, Mtime, ctime differences and how to reduce atime updates

https://mp.weixin.qq.com/s?__biz=MzA3MzYwNjQ3NA==&mid=400742435&idx=1&sn= d5fa0776ebce30be1de897cd7d1eb295&scene=0&key= D72a47206eca0ea9ec5b7c574d593f1cec476c38b6d36e28bf256e4822e7188e92f5b6b9c7bb0398b7e86bad90f786a0&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.1+build (15B42) &version= 11020201&pass_ticket=fm72ikhe73ohijfh1jkrnofjhmnntunq8yrmtlceass4plhwdpuno9ncxgwvzjjc

Important: Why do we need to use the Noatime option when we mount a disk? What do these hours mean? This article tells you the answer.

2. How do we put the continuous deployment of simplified

https://mp.weixin.qq.com/s?__biz=MzA4NjAzMjEyOA==&mid=402441122&idx=1&sn= 8195b678bfeaf2b4c153afecbf43c5a9&scene=0&key= D72a47206eca0ea962fd5dff5f17c807cf7ef4c67efe90bb83ef42de5ffefdd5c2bc22225fe8cfe134253cda75bb31fe&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.1+build (15B42) &version= 11020201&pass_ticket=fm72ikhe73ohijfh1jkrnofjhmnntunq8yrmtlceass4plhwdpuno9ncxgwvzjjc

Important: When it comes to continuous deployment people often think of this as an extremely complex system, this article starts with the RPM package, grasps the key link to simplify, designs and realizes a set of visual continuous deployment system.

3. three images see Linux performance monitoring, testing, optimization tools

https://mp.weixin.qq.com/s?__biz=MzA3MzYwNjQ3NA==&mid=400775633&idx=1&sn= 4f8fe1e484d1f2bb9843fca333a4340a&scene=0&key= D72a47206eca0ea954e2eb2b0436b5df588936fb64d5ae5a02d42b3f5af2f5d5589f866dbab120b4319030a3971bd445&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.1+build (15B42) &version= 11020201&pass_ticket=cdp7crtptmbyombyxm9%2b2omffnjuj%2b%2fb%2f%2f7dsgvkouc3uthrqzrli9b6te3shfc6

Point: Look directly at the picture, not much to say.

4. Docker Monitoring Combat

Http://news.oneapm.com/docker-monitoring/?hmsr=toutiao.io&utm_medium=toutiao.io&utm_source=toutiao.io

Important: monitoring is always an essential component of service operations. With the popularization of Docker and container technology, the corresponding monitoring needs to keep up with the times, this paper introduces some monitoring solutions based on Docker, especially the open source monitoring system of Prometheus. By the way, Prometheus claims to be the closest surveillance system to Borgmon. (I have not used it for reference only)

Tools Collection

1. Singleton mode (Singleton) and its C + + implementation

Http://zkt.name/dan-li-mo-shi-singleton-ji-c-shi-xian/?hmsr=toutiao.io&utm_medium=toutiao.io&utm_ Source=toutiao.io

Key points: Several implementations and advantages and disadvantages of C + + singleton mode.

2. The wisdom of programming

Http://www.jianshu.com/p/7645a5ea7f46?hmsr=toutiao.io&utm_medium=toutiao.io&utm_source=toutiao.io

Important: I agree with many of the points in the article, and from now on, please be careful to write clear and readable code.

3.28 Unix/linux Command-line artifacts

http://mp.weixin.qq.com/s?__biz=MjM5OTMxMzA4NQ==&mid=402444984&idx=1&sn= c991ad907e4e787fb5dc381b96296a97&scene=1&srcid=1122ieisaqn3htdbc26fskdy&key= D72a47206eca0ea9fd7f7a51e1b0e50e480cb76589c31d9dc525f9888b9b7021cda5da8b21b0ff6adf8039edace06305&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.1+build (15B42) &version= 11020201&pass_ticket=fm72ikhe73ohijfh1jkrnofjhmnntunq8yrmtlceass4plhwdpuno9ncxgwvzjjc

Important: A lot of useful tools, especially performance detection related.

3. Linux operations more useful tools

Http://blog.jobbole.com/95242/?hmsr=toutiao.io&utm_medium=toutiao.io&utm_source=toutiao.io

Key points: more practical tools, such as Nethogs, Iotop, Htop, and so on, unfortunately the company's standard environment did not, tracing the problem on the line has to install itself.

4. deep Analysis of C + + object Pool Automatic recovery technology implementation

http://mp.weixin.qq.com/s?__biz=MzAxNDI5NzEzNg==&mid=400869339&idx=1&sn= 970eaca55334797d79738b4fea77609f&scene=0&key= Ff7411024a07f3eb0f6bdf59b9e5fc1ba0a9678f9c1ac3a3bc2914a14008ae7a5fe58ade07b9565cf8d7027cafe2af6b&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.1+build (15B42) &version= 11020201&pass_ticket=9mb1xkc0hqedpjfc7tyecllaequc%2bjr0miadlbozrmw41%2fd9p16mwnwl3kh29aze

Important: Use C + + to implement the object pool code, using C++11 technology.

5. RAID level structure Knowledge analysis

http://mp.weixin.qq.com/s?__biz=MzAwNjQwNzU2NQ==&mid=400638185&idx=3&sn= 5ef5b18bee92fda8c5f7ef9c7bcd7dd8&scene=1&srcid=1129byn23iruu86lagfwtdm9&key= Ff7411024a07f3ebb3a7a9c6cc87ba620c1d57ea1c035dcd5420c9ffa524e5e426db0dd2638240b5d37b90eb3be4e265&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.1+build (15B42) &version= 11020201&pass_ticket=9mb1xkc0hqedpjfc7tyecllaequc%2bjr0miadlbozrmw41%2fd9p16mwnwl3kh29aze

Important: There is no way to get dizzy by the various RAID modes in the machine installation template. After reading this article, all RAID levels are in your hands.

A weekly technical update on distributed technology 2015.11.29