Case study of HBase optimization: FacebookMessages System Problems and Solutions

HDFS was designed to store large files (such as log files) for batch processing and sequential IO. However, the original intention of the HBase design built on HDFS is to solve the random read/write requests of massive data. How can we combine the two components with the opposite original intention? This hierarchical structure design mainly aims to make the architecture clearer, HBase

HDFS was designed to store large files (such as log files) for batch processing and sequential I/O. However, the original intention of the HBase design built on HDFS is to solve the random read/write requests of massive data. How can we combine the two components with the opposite original intention? This hierarchical structure design mainly aims to make the architecture clearer, HBase

HDFS was designed to store large files (such as log files) for batch processing and sequential I/O. However, the original intention of the HBase design built on HDFS is to solve the random read/write requests of massive data. How can we combine the two components with the opposite original intention? This hierarchical structure is designed to make the architecture clearer. The HBase layer and HDFS layer perform their respective jobs, but it brings potential performance degradation. In many business scenarios, the two most complained about using HBase are Java GC and random read/write performance. Facebook Messages (hereinafter referred to as the FM System) system is the first case of HBase in the online storage scenario (Apache Hadoop Goes Realtime at Facebook, SIGMOD 2011 ), recently, they published a paper Analysis of HDFS Under HBase at the top storage Conference FAST2014: A Facebook Messages Case Study analyzes some of their problems and solutions in using HBase. For more information about using HBase for online storage, see.

This paper first introduces Facebook's analysis methods, including tracing/analysis/simulation, FM system architecture, file and data composition, etc. Next we will analyze some performance problems of the FM system, the solution is also proposed.

Http://www.infoq.com/cn/articles/hbase-casestudy-facebook-messages

Original article address: HBase optimization case analysis: Facebook Messages System Problems and Solutions. Thank you for sharing them with me.