A weekly technical update on distributed technology 2016.04.17

Distributed System Practice

1. The essence of architecture is management complexity, and microservices themselves are the result of architectural evolution

http://mp.weixin.qq.com/s?__biz=MzA5Nzc4OTA1Mw==&mid=411129391&idx=1&sn= Ebf06fb5cc4a5f57f86341ba4114cab8&scene=0#wechat_redirect

Important: This article introduces the microservices architecture practices of well-known internet companies such as Ctrip, Netflix and SoundCloud, and concludes that the evolution of the microservices architecture is essentially a reflection of the complexity of the management architecture.

2. Consistency hash Algorithm detailed

Http://www.jianshu.com/p/e8fb89bb3a61?hmsr=toutiao.io&utm_medium=toutiao.io&utm_source=toutiao.io

Important: If you do not know the consistency hash algorithm, then this article is for you to prepare. After reading, you will have a basic understanding of the consistency hash algorithm.

3. BFS: Small file storage System supporting Bilibili

https://mp.weixin.qq.com/s?__biz=MzAwMDU1MTE1OQ==&mid=406016886&idx=1&sn= f5aa286373fb981c9de904568fe7ddb2&scene=1&srcid=0411e8naa5e9o1r10u1utv6y&key= B28b03434249256b155024dc0c2ce94fdb443b81f70f8d5412658febd9bb7ee4843920e5f0b0314a1a2a1d4a301bbae4&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.4+build (15E65) &version= 11020201&pass_ticket=dywex3am4skhkqbs6%2bkkmtkavy%2fuibuueocr5aqbi5xwrop7uql1t6dx1bdpl%2fau

Key points: This article introduces the Bilibili self-developed picture storage System architecture and design, the whole design in a simple, reliable, stable principle, the whole design is not particularly complex technology, the structure read is very easy to understand, whether it is operation or problem tracing, are relatively easy, This is where we should learn.

4. Hadoop NameNode Highly available (high availability) Implementation resolution

http://www.ibm.com/developerworks/cn/opensource/os-cn-hadoop-name-node/?hmsr=toutiao.io&utm_medium= Toutiao.io&utm_source=toutiao.io

Key points: The Namenode state of HDFs is a very important information, how to achieve a master multi-slave Namenode is a key factor to improve the stability of HDFs under the premise of guaranteeing consistency. This article describes the shared state storage Namenode based on QJM (Quorum Journal Manager), and the fact that the INF also implements a relatively generic QJM component.

Service and resource management technology

1. Create a cross-host container network using open vswitch in Docker

https://mp.weixin.qq.com/s?__biz=MzA5OTAyNzQ2OA==&mid=2649689779&idx=1&sn= 11326ff26ec69abe6890db7434360f8b&scene=0&key= B28b03434249256b73650d69c4a66bac98b606e40bbf227ee76ae0454084b3e25cff67d33044918315b01b2d6ae577ad&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.4+build (15E65) &version= 11020201&pass_ticket=dywex3am4skhkqbs6%2bkkmtkavy%2fuibuueocr5aqbi5xwrop7uql1t6dx1bdpl%2fau

Important: This article describes how to use open vswitch to provide network support for Docker 1.9.0 and later versions. The network part is the most complex place of the container, with open vswitch to flexibly manage the network of the container.

2. Kubernetes Schedule

Http://www.skycloudsoftware.com/index.php/2016/04/12/kubernetes003.html

Key points: This article introduces the Kubernetes scheduler in detail from the scheduling process, the scheduling algorithm and the resource limitation, which is very helpful for the deep understanding of the Kuberntes scheduling function.

3. Monitoring technology for Docker containers

https://mp.weixin.qq.com/s?__biz=MzI4NTA1MDEwNg==&mid=403252900&idx=2&sn= e0a7de8b86e547eef8cf8343d2590d3d&scene=1&srcid=0413nhtaexmlv6bn7mvapggh&key= B28b03434249256bcb435176a2032b48ca99480570bff5be9c764b3f554a5fdfe84107f81b5a85aa03752f84fc76f364&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.4+build (15E65) &version= 11020201&pass_ticket=dmeemr4uazvypn%2bqb9sxkn7zhbj4sxp%2bwxjeq3i42gfibshmpv%2bun9chszpar9wx

Important: Services cannot be separated from monitoring, this article describes several common Docker-based monitoring solutions, and recommends a more commonly used tool portfolio.

4. What are the notable changes to Docker 1.11?

https://mp.weixin.qq.com/s?__biz=MzI3OTEzNjI1OQ==&mid=2651492598&idx=1&sn= a370bcd184db26db5b4d384dcf0b7f2d&scene=0&key= B28b03434249256bfb04e9c3d3a8f3cf56b7404ea7ed79f54aea4cada1c85b981700fa53b90f10f43390b719c6fdd301&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.4+build (15E65) &version= 11020201&pass_ticket=p%2fkcyxikbs9m%2fy6pjzes8n2a56adxh0slt6poaxfmwme6h8m4kakdkmq80vvpmep

Important: Docker 1.11 was released on April 13, the release of Runc and Containerd for the first time, and a new logo was designed for Containerd. In addition, there are some other new features, you see the article understand it.

Service High Availability Technology

1. Analysis of load balancing algorithms in distributed systems under high availability scenarios

http://tech.youzan.com/load-balancing-algorithm/

Important: This article introduces several common load balancing strategies, and emphatically analyzes the request success probability of the minimum concurrency policy, which is instructive for us to design load balancing strategy.

2. The fault-tolerant construction practice of QQ member billion level Web system

https://mp.weixin.qq.com/s?__biz=MjM5MDE0Mjc4MA==&mid=403476561&idx=2&sn= d68cebba4ff4d8f0eda4e62e8fb4b614&scene=0&key= B28b03434249256bb79d85e42a0da423a5108a839ef8450b04e78556b636ad98525790c35241f0832dd06c5b78f7dbef&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.4+build (15E65) &version= 11020201&pass_ticket=dywex3am4skhkqbs6%2bkkmtkavy%2fuibuueocr5aqbi5xwrop7uql1t6dx1bdpl%2fau

Important: This article details the stability of the QQ membership system, including load balancing, retry, downgrade, overload protection and other strategies, and our strategy has a lot of similar places.

3. The practice and experience of high availability systems in public reviews

https://mp.weixin.qq.com/s?__biz=MzAwMDM2NzUxMg==&mid=2683177565&idx=1&sn= bbe68683ea4c0409b6886594b829464d&scene=0&key= B28b03434249256b52571ba82d8b215eb9fdc3f07951919709ca95cb3c9c987bac8dbf32dc6282ea67675ab408308d6a&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.4+build (15E65) &version= 11020201&pass_ticket=mtgg3i2llfmjhrzkobqdhiup9fovjnhq7qtwjdz3dq5m%2bc2sh4jhuurfzigeffpb

Key points: This paper introduces the system evolution process of public reviews, especially the high-availability solutions of different forms practiced in the process of evolution.

DevOps Technology

1. Google's production environment package management system

https://mp.weixin.qq.com/s?__biz=MzI4OTE3MTI1NA==&mid=402328249&idx=1&sn= 0ae1a016b26adce549fdffc919dbeb54&scene=0&key= B28b03434249256bb4659ccdf99d4f7703a9be29ce9c7c34f27472cb7d28843a59b2581f3c1ffe7efe8695ab3341789b&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.4+build (15E65) &version= 11020201&pass_ticket=dywex3am4skhkqbs6%2bkkmtkavy%2fuibuueocr5aqbi5xwrop7uql1t6dx1bdpl%2fau

Important: How to efficiently produce and distribute packages on millions of server sizes? I believe many of our classmates are plagued by different environments and configurations, so let's see how Google solves this problem.

2. Talk about monitoring (a): What is worth monitoring and the choice of monitoring indicators

https://mp.weixin.qq.com/s?__biz=MzA5Nzc4OTA1Mw==&mid=412113314&idx=1&sn= b500ceec76a6a844fa7d0c277c2d166b&scene=0&key= B28b03434249256b71f1c5ac9b7234b49f150967f118d697a1b9c219cfe73a5171a9eb9caef952f55c3246e9165e520d&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.4+build (15E65) &version= 11020201&pass_ticket=p%2fkcyxikbs9m%2fy6pjzes8n2a56adxh0slt6poaxfmwme6h8m4kakdkmq80vvpmep

Important: This article is the first of a series of "chat and monitor" articles, and introduces the relevant content of monitoring indicator trade-offs.

3. The road of operation and maintenance automation based on CMDB and Saltstack

https://mp.weixin.qq.com/s?__biz=MzA3MzYwNjQ3NA==&mid=2651296412&idx=1&sn= 645b8d9f5ee9226fcf57987a7fac95c7&scene=0&key= B28b03434249256bc04268e50009d90376b775d44b7c3d4da020267bb4a6bf0852b12fd54d8b9345270a2cdd6190f804&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.4+build (15E65) &version= 11020201&pass_ticket=p%2fkcyxikbs9m%2fy6pjzes8n2a56adxh0slt6poaxfmwme6h8m4kakdkmq80vvpmep

Key points: This paper summarizes the practice of automatic operation and maintenance system based on CMDB and saltstack, and it involves the experience of system planning, implementation route, product design and architecture design.

Hodgepodge

1. How to improve the performance of glog 10 times times

http://mp.weixin.qq.com/s?__biz=MzA4MjA0MTc4NQ==&mid=403091448&idx=1&sn= E0a52f52ee3a4b1c511cb7f84faa63ad#rd

Key points: Some tips for improving glog performance

2. C + + template techniques

1190000003901666

Important: Some advanced tips for C + + templates, including Type_traits, SFINAE, CRTP.

3. Architecture knowledge required for four stages of technology growth

https://mp.weixin.qq.com/s?__biz=MzAwMDU1MTE1OQ==&mid=406063297&idx=1&sn= 0cd1fc5495e1e26471d7a4ffcd96cd0f&scene=0&key= B28b03434249256b6fa2d245c6016d5182ee7d9b8a48b0d704c373c6861b3f69b999a04d1c7a7a4773f4cd9ff62d246d&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11.4+build (15E65) &version= 11020201&pass_ticket=mtgg3i2llfmjhrzkobqdhiup9fovjnhq7qtwjdz3dq5m%2bc2sh4jhuurfzigeffpb

Important: This article is an analogy to the Saint-Warrior upgrade process, which explains the 4 phases of the architecture engineer's technological growth, as well as the architectural knowledge required at each stage to help you plan your own development direction.

4. Git uses the canonical process

Http://www.ruanyifeng.com/blog/2015/08/git-use-process.html

Important: Git is very flexible to support the branch and the local warehouse, so based on a lot of git play, here is recommended a more reliable play, for everyone to reference.

A weekly technical update on distributed technology 2016.04.17