I am concerned about the week of technical developments 2015.10.18

Distributed System Practice

1. Learning streaming real-time distributed computing design from Storm and spark

Http://www.csdn.net/article/2014-08-04/2821018/1

Point: Stream computing is not something new, I believe many students have used. However, before stream computing is often used in business-related areas, with the urgent need of large-scale distributed systems for trace and metric data collection, complex data analysis and aggregation functions can be realized based on time series database and streaming computation, this article helps you understand the principle of streaming computing. As you can imagine, if you want real-time statistical performance to consume the first 100 BS, how to use streaming computing modeling.

2. Google File System

Http://thu-cmu.cs.tsinghua.edu.cn/curriculum/dscourse/slides/01Overview.pptx

Important: This week, we'll revisit Google's classic paper, GFs. Although GFs was published in 2003, many technologies still seem to be valuable to date, such as lease mechanisms, checksum mechanisms, conformance models, high availability, and so on. Link is not the paper itself, but Tsinghua University's distributed System tutorial, with GFS as the first lecture.

Service and resource management technology

1. Google uses Borg for large-scale cluster management (thesis translation)

https://mp.weixin.qq.com/s?__biz=MzA5OTAyNzQ2OA==&mid=208440555&idx=1&sn= 113e883b818f430ce5ee68423cecc0d5&scene=0&key= 2877d24f51fa5384c503efd3df4dba0864707a9514e7f7e8d92b366f986c1e5a03a86624dad996e66fd621f8d590da84&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11+build (15A284) &version= 11020201&pass_ticket=zedzfiqex7uvcublcyi%2feefvs8l8xyihdqeapzhvc5ns3jeccig6ngnngqvwwmt9

Main points: The paper is long, the article is still in the continuous translation. Recently, the Beehive team has discussed the Borg paper in depth, and interested students can discuss it with us.

2. Comparison of five Docker monitoring tools in "combat"

http://dockone.io/article/397

Important: Monitoring is one of the infrastructure that has always been PAAs, this article describes the 5 container performance collection and monitoring systems based on Docker. In addition, HTTP://DOCKONE.IO/ARTICLE/171 introduced here Prometheus (Prometheus) is also a good choice.

3. Docker vs. virtual machine performance Comparison

Http://chenbiaolong.com/2015/02/26/dockervm/?hmsr=toutiao.io&utm_medium=toutiao.io&utm_source= Toutiao.io

Important: We've always said that Docker is a lightweight virtualization solution, and this article compares the CPU compute power and memory access to two dimensions, to what extent Docker and virtual machines are lightweight.

4. CPI2:CPU performance isolation for shared compute clusters

Http://eurosys2013.tudos.org/wp-content/uploads/2013/paper/Zhang_2.pdf

Important: Although the cgroup mechanism can solve the problem of resource isolation to a large extent, the competition of resources at the lower level still exists. A lot of research work is done at the kernel level to enhance the resource isolation mechanism, but this article from the point of view of the problem, the use of cyclical data collected by the CPI, CPU competition to detect the situation, and find ways to eliminate (adjust the affected job CPU quota or migration job). From the paper, it is shown that the response time of the CPI and the service of leaf Service and intermediate service (BC/BS) is better linear, but the root node (AC/US) can not be linearly corresponding, because the CPI is more the CPU resource consumption than the network and IO. The ideas and conclusions of this article are very instructive to the bc/bs of our detection anomaly, expecting the retrieval system to open the resource isolation and realize the day of mixing.

5. the practice and thinking of enterprise-level cloud platform

Http://dockone.io/article/740?hmsr=toutiao.io&utm_medium=toutiao.io&utm_source=toutiao.io

Important: The "editor's word" Guest is engaged in the development of many cloud platforms. In this article, he will combine his experience to share the enterprise-class cloud application platform practice and thinking about content.

6. What should I run in the container?

https://mp.weixin.qq.com/s?__biz=MzA5OTAyNzQ2OA==&mid=208468120&idx=1&sn= 1dd37080ada7ea9eb4963428abda108d&scene=0&key= B410d3164f5f798e467ba918f24dc886aadb91d232b8a334dd7de697e10499f78ae5335061083e085ab2d83ca42d76d7&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11+build (15A284) &version= 11020201&pass_ticket=jyimr5n3imb%2fjboi9vosvnvnws91eo5dryet%2btckryesdp7bhihkyutpob1qjojx

Important: The concept of containers has been more and more accepted by more people, but after the container, what should we put in the container, what is placed outside the container? This article provides some points of view.

7. Practice | The way of the structure selection of Otto Otto Group

http://dockone.io/article/737

Important: The development of the container technology has promoted the development of micro-service architecture, the flexibility of micro-service architecture, and the characteristics of change efficiency have attracted many company's architecture team. Otto.de is an online shopping website in Germany, the first half of this article mainly introduces several system architectures and their advantages and disadvantages, the latter part mainly explains the otto.de of the micro-service architecture.

Service Scheduling and Trace technology

1. The secret of Time series database

Http://www.infoq.com/cn/articles/database-timestamp-01?utm_campaign=rightbar_v2&utm_source=infoq&utm_ Medium=articles_link&utm_content=link_text

Important: The time series database is the basis for supporting massive metric data collection and analysis, and this article explains not only why traditional NoSQL data does not support the storage and querying of metric data, but also explains why Elasticsearch can support it. What we need here is a search engine, not just the KV class of NoSQL databases.

2. Google launches mobile Web acceleration project amp

Http://www.infoq.com/cn/news/2015/10/Google-AMP

Important: How to improve the performance of Web pages attracts a lot of attention from engineering staff and researchers. To solve this problem, Google recently launched a mobile web acceleration project (accelerated Mobile pages, AMP) with dozens of technology organizations and companies. Unfortunately, the official website of the project will not open, need FQ.

3. Linux CFS Scheduler _ Theoretical Model

Http://www.cnblogs.com/openix/archive/2013/08/13/3254394.html

IMPORTANT: CFS is a fair scheduling algorithm supported by Linux self-2.6.23 kernel, and introduces the concept of process virtual running time, so the design of the whole scheduling algorithm is very simple. There are a lot of articles about CFS, but I think this is the best one because there are simple examples to help you understand it. In fact, the CFS algorithm thought can also be used in our daily development, many times, we will encounter the "Priority selection task" scenario, such as Beehive's data Tranfer task selection strategy, this time may wish to think of the CFS algorithm ideas. CFS paper: http://landley.net/kdocs/ols/2010/ols2010-pages-245-254.pdf

DevOps Technology

1. Build a Lean operations organization for IT performance

http://mp.weixin.qq.com/s?__biz=MzA4Nzg5Nzc5OA==&mid=208069436&idx=1&sn= 868db41c1ad3487f1abf594f7bf26148&scene=0&key= 2877d24f51fa53847726f76f9085ccd96166fbe7821c1fe030c40855b47fdf8736e1ed981d198cedca60f1e92d22818e&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11+build (15A284) &version= 11020201&pass_ticket=f0ryv30xthr4cvjlyaey8jzpoegw2qbjqc15s6xl8rurtkjnibheufas29krfdwe

Key points: How to improve the efficiency of IT operations, this article from the service delivery, failure recovery and other aspects of this issue.

2. Honor or hardship? | Also on full stack engineers and DevOps

http://mp.weixin.qq.com/s?__biz=MzA4Nzg5Nzc5OA==&mid=208086642&idx=1&sn= 2cd6fc667af1bd69b3c6014eefd25ba7&scene=0&key= B410d3164f5f798e6cba7f15c09f6d1479ae3e56972bfb5469c7964f12737a8bed962e8dfbd5bd6dc6035d2e596b9c09&ascene=0 &uin=mjk1odmyntyymg%3d%3d&devicetype=imac+macbookpro11%2c4+osx+osx+10.11+build (15A284) &version= 11020201&pass_ticket=jyimr5n3imb%2fjboi9vosvnvnws91eo5dryet%2btckryesdp7bhihkyutpob1qjojx

Main points: Opinion, for reference only.

Tools Collection

1. instructions for using the Linux Mount Mount device

http://blog.jobbole.com/92695/

Important: The mount command may be familiar to many OP students, and this article describes the detailed use of the Linux Mount command to help you understand how Linux manages the device.

2. MAC Setup Guide for obsessive-compulsive disorder

Https://github.com/macdao/ocds-guide-to-setting-up-mac?hmsr=toutiao.io&utm_medium=toutiao.io&utm_ Source=toutiao.io

Important: Very comprehensive MAC work and development environment settings

3. Introduction to Linux awk built-in variable usage

Http://www.cnblogs.com/chengmo/archive/2010/10/06/1844818.html

Important: awk built-in variable usage

4. Best Learning Resource for beginners in Python

Http://python.jobbole.com/82399/?hmsr=toutiao.io&utm_medium=toutiao.io&utm_source=toutiao.io

Key points: Picking

I am concerned about the week of technical developments 2015.10.18