Jame 'sreading07-25

Reading and analyzing papers t. the data deduplication of low-cost virtual machines in the cnzHFP5Yj cloud storage environment, HongTang and. tom's former colleague is writing a thesis jointly by TaoYang at Santa Barbara Berkeley. For the backup method (ShardingMeta information) in the cloud environment, the fingerprint is calculated first and then duplicated, although the overall algorithm is not improved

Reading and analyzing low-cost virtual machine data deduplication in http://t.cn/zHFP5Yj cloud storage environment, Hong Tang and. tom's former colleague, Tao Yang from Berkeley Santa Barbara, co-authored the paper. For the backup method (Sharding Meta information) in the cloud environment, the fingerprint is calculated first and then duplicated, although the overall algorithm is not improved

Reading and Analysis of papers

Low-cost virtual machine data deduplication in http://t.cn/zHFP5Yj cloud storage environment, Hong Tang and. tom's former colleague, Tao Yang from Berkeley Santa Barbara, co-authored the paper. For the backup method (Sharding Meta information) in the cloud environment, the fingerprint is calculated first and then duplicated, although the overall algorithm is not improved much, the overall efficiency and overhead are not bad due to splitting + parallel processing.

Http://t.cn/zHF7hqo Facebook released a revised version of RS code to ease network bandwidth problems caused by traditional RS code recovery, in the basic 10 + 4 RS code mode, on average, they consume 30% TB of network bandwidth every day for data recovery. Using the new modified RS code can theoretically reduce the network bandwidth requirement by about, the main idea is to include a copy of the data with the previous strip when calculating the verification code.

Http://t.cn/zQZCowQ Murat Demirbas's interpretation of Google Spanner Paper (there are a lot of classic papers in the blog), focuses on the implementation and role of this article TrueTime API, implementation: rely on atomic clock, compare the Paxos and 2 PC Prepare timestamps to obtain the Snapshot Time. The function is to simplify the Snapshot Read design and implement a Snapshot Time similar to the Oracle Time-> SCN.

Several good theme recommendations on http://t.cn/zHFKAKr Velocity 2013, in addition to the small part that has been posted a few days ago, as well as Performance Methodologies for Production Systems (Brendan Gregg), Quantifying Abnormal Behavior (Baron Schwartz ), A Systematic Approach to Capacity Planning in the Real World (Twitter)

Statistical and Monitoring Analysis

Http://t.cn/zQZNFxc Baron Schwartz's new company blog explains four basic rules for statistical process control, 1. there are indicators beyond the range of three standard deviations, 2. two of the three consecutive points are between 2-3 standard deviations, 3. four of the five consecutive points are out of two standard deviations, 4. 9 consecutive points on the average side. etsy's Skyline (http://t.cn/zqznfxvand oculusrefer to the statistical process control methodology.

Http://t.cn/zQqS4OT why the average is not good, while the percentile (percentile) is very useful. the avg, min, and max charts are displayed at the same time in the figure. No charts show the latencies of 50%, 75%, 90%, and so on, the latter can significantly improve the efficiency of Problem Analysis/locating.

Design and use of products and systems.

Http://t.cn/zQZCDkp Lars Hofhansl introduces a design defect in HDFS (same as HBase). In extreme cases, when the data center suddenly loses power, HBase may not only lose the latest data, if Compact is performed again, the data updated earlier may also be lost. In this article, their solutions are provided by adjusting the dfs parameter. datanode. sync. behind. writes and dfs. datanode. synconclose

Interview with Oracle NoSQL database in http://t.cn/zQZpWtF. focuses on the Major/Minor Key design (I personally prefer this design, very close to DB Sharding); Master/Slave Replication design, ensure write consistency through Paxos and simple majority arbitration; Support for avro serialization (better Json Integration); Support for simple conversion into Oracle External tables, facilitating data exchange.

Http://t.cn/zQZWRaz to get quiet (no output) Slow Query log in terms of scalability is more happy. the author's idea is actually very simple. Most of the scalability problems come from insufficient Query efficiency in databases, especially the unreasonable design of Query indexes. most companies can solve Scalability through better SQL design and better index design. there is also a small part that needs to be matched with Sharding.

Technical understanding and methodology Introduction

Http://t.cn/zQbKq2g from the Brendan Gregg perspective, to become an expert of some basic principles: 1. rigorous, 2. there is no difficulty in the world, but in terms of time, there is a price, 3. use scientific methods and pay attention to their assumptions. 4. do not trust everything (easily), especially for stress testing. OS indicators will also lie. note the categories of known knowns, known unknowns, and unknown unknowns.

Http://t.cn/zQbOYj8 Lessons from Building and Scaling LinkedIn By Jay Kreps. many experiences are of reference significance. 1. most Scale systems are related to Scale State (or database storing State). 2. how to Scale internal development capabilities 3. how to Scale the system Scale 4. how to manage the SOA-based services of Large scale ).

Http://t.cn/zQ55h2U Theo believes that when Scale Up is feasible (to meet the needs of the next 1-2 years), should not do Scale out. if your system/project growth rate is lower than Moore's Law, you should always consider using larger machines (better cheap pcs) to meet your needs. scale out requires a lot of engineer resources to solve infrastructure problems, and engineer resources should be used in more efficient places.

To be truly excellent, one must treat it as a craft. one must become a craftsman. through experience learn discipline. and through practice achieve excellence. By Theo. A Career in Web Operation
Step 1, educate yourself, step 2, be disciplined, step 3, learn from & share with your peers, step 4, be patient. experience takes time (and mistakes ). everyone in your organization needs Operational Mentality. operations is a state of mind it is a state of being it is a mentality.

Http://t.cn/zQZYxn5 If you want to get the factor 50 speed-up of SSDs, you 'd better avoid reading large chunks of sequential data, because that's where you can only gain a factor five improvement. A very insightful sentence.

Social Sciences

Http://t.cn/zHs11Jl [Zhou Qiren] is still not "fair", because there will be a growing population in the future. No matter whether it is for the new generation or the people who are down to the village, the stock of wealth not involved in the allocation of wealth will inevitably lead to wealth inequality between villagers and families in the future. Let's wait. For the sake of the future's "Starting Point of fairness", we have to wait for the birth of an everlasting and equitable land system in the midst of endless conflicts.

Http://t.cn/zjz0p4F "a particular social structure determines whether a set of statements are lie or knowledge. In other words, knowledge is not objective. It first depends on the power structure of the society in which it belongs ". the general idea of Fei xiaotong is: what kind of social structure you have, and what kind of knowledge structure you have accumulated. [Wang Dingding: lies and freedom of knowledge are the whole thing]

Social Science Book recommendation

Dan. from the perspective of behavior, why are we dishonest? In what scenarios are we dishonest, how can we reduce people's dishonesty through rule control.

The founder of Jason Fried "come back" 37Signals introduced how to develop and process the system through simple thinking. For more details, refer to the blog post on @ left ear mouse.

In the art of conscious thinking, Rolf dompeili lists 52 thinking traps that humans often make mistakes, that is, some defects in human cognition, reading more helps to correct your own thinking flaws.

Related posts:

1. Jame's Reading 10-14
2. Jame's Reading 09-10
3. Jame's Reading 06-28

Original article address: Jame's Reading 07-25. Thank you for sharing it with me.