hbase performance monitoring

Performance monitoring tool: 1. Task Manager 2. Performance Logs and alarmsI. Task Manager1. Purpose: display applications running on the system Program Process, CPU, memory, and other usage and statistical information.2. Open Mode (local or remote)1> right-click the blank area in the taskbar and select "Task Manager"2> CTRL + ALT + del ctrl + ALT + end3> Ctrl +

Common Linux system performance monitoring commands and linux system performance monitoring URL: http://www.cnblogs.com/myresearch/p/linux-monitor-tools.html.Monitor CPU usage Run the following command: [root@localhost ~]# gnome-system-monitor The graphical tool GNOME System Monitor is displayed, as shown in: Monitor C

Reprint: http://ju.outofmemory.cn/entry/50064With the drive of Big data table application, our hbase cluster is getting bigger and larger, however, due to some uncertainties in machine, network and HBase, the system is faced with some uncertain faults.Therefore, HBase has a lot of region components and needs to control the state of each table's region. Analysis:

Introduction to common linux performance monitoring tools and linux performance monitoring tools Record several frequently used ones.1. top/htop As we all know about top, there are several common parameters: (you can also start and interact) -D: Set the delay time-S: accumulative time mode k: End Process q: Exit topZ:

. The benefit of this is that data recovery can be done after the regionserver outage. Therefore, for relatively less important data, you can discard the write Wal log by calling Put.setwritetowal (false) or the Delete.setwritetowal (false) function when Put/delete operation, This improves the performance of data writes. It is important to note that it is prudent to close the Wal log, because then, once Regionserver is down, put/delete data will not b

single request. When setting this value, the main reference is monitoring memory. Note that if the number of region servers is small and a large number of requests are on the same region, the read/write lock caused by memstore triggering flush will affect the global TPS, the higher the number of IO threads, the better. Enabling RPC-level logging is enabled during stress testing to monitor the memory consumption and GC status of each request at the sa

ordered by executions:The top SQL that is sorted by the number of executions of SQL is recorded. This sort shows the number of SQL executions that are within the scope of the monitoring.4.6 SQL ordered by Parse Calls:Top SQL that records the number of soft parse times for SQL. When it comes to soft-parsing (soft prase) and hard-parsing prase, it's impossible to say how Oracle handles SQL.4.7 SQL ordered by sharable Memory:The top SQL that SQL occupie

....00000120120902000001000001201209040000020000012012090600000300000120120908000004000001201209100000050000012012091400000700000220120912000006000002201209160000080000032012091800000900000420120920000010How to use this watch?After setting up a scan object, we Setstartrow (00000120120901), Setendrow (00000120120914).In this way, scan only scans the userid=1 data, and the time range is limited to the specified time period, which satisfies the filtering of the results by user and by time range. A

We often see articles boasting about how fast a product is, how strong it is, and how it is better to test than some of the data described. The reason may be that you don't really understand the internal structure, and are not aware of its performance tuning approach. This article transferred from Taobao's Ken Wu classmate's blog, is currently seeing the more complete hbase tuning article. Original link:

org.apache.hadoop.hbase.client.Scan;import org.apache.hadoop.hbase.io.ImmutableBytesWritable;import org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil;import org.apache.hadoop.hbase.mapreduce.TableMapper;import org.apache.hadoop.hbase.mapreduce.TableReducer;import org.apache.hadoop.mapreduce.Job;import org.apache.log4j.Logger;import java.io.IOException;public class HBaseMapReduceDemo { static Logger LOG = Logger.getLogger(HBaseMapReduceDemo.class); static class Mapper1 extends TableMapp

For HBase in single column family single column qualifier and single column family multiple column qualifier two scenarios, batch put write performance comparison The following is a combination of HBase source code to explain the phenomenon of simple analysis. 1. Test results When the client batch writes, the TPS and RPC times of single row and Single-column gr

BottomFour years hbase. Make a preliminary test write performance on the on-line machine hbase cluster. The following specific instructions to do the test content.DescriptionHBase surrounding environment0.96 version number, 8 region servers. Default configurationWrite Data descriptionSingle column family. The value of two column qualifier is string + random 8-bit

configuration of the server directly, it is expected that the more clusters, the higher the number of server CPU, the more concurrent processing power.Pre-partitioningHregion is the smallest unit of distributed storage and load balancing in HBase. The smallest unit means that different hregion can be distributed on different hregion servers. However, a hregion is not split across multiple servers. Hbase.hregion.max.filesizeHstoreFilethe maximum value

It is also necessary to perform basic monitoring on a single server or VPS. In the past, Zhao Rong did not monitor VPS, because when some servers were added, his performance became even more tense. There are also a lot of foreign server performance monitoring websites, but it is a little difficult for people like Zhao

System optimization is a complex, complex, and long-term task. monitoring, collection, testing, and evaluation are required before optimization. After optimization, testing, collection, evaluation, and monitoring are also required, it is also a long-term and continuous process. It is not to say that the optimization is now done, the test is done, and it can be done once and for all in the future. It is not

++){ byte[] Rowkey = Bytes.tobytes (Randomstringutils.random (8,"abcdesssss")); byte[] value = Bytes.tobytes (Randomstringutils.random (Ten,"Iojkjhhjnnbghikklm")); Put put=NewPut (Rowkey); Put.add (Bytes.tobytes (FAMILY_CF), Bytes.tobytes ("value"), value); Puts.add (Put); if(i%10000==0) {table.put (puts); Table.flushcommits (); Puts.clear (); }}3.3. Self-Added columnsDesttable.incrementcolumnvalue (Rowkey, Bytes.tobytes (FAMILY_CF), Bytes.tobytes ("testincrement "), Long.parselo

is divided evenly into three parts, the eldest brother occupies two, and the younger generation occupies a copy. Reference Sun Java System application Server Enterprise Edition 8.2 Performance Tuning guide), this ratio is not suitable for all situations, especially when local variables in your application are far greater than the global variables, and a large number of local variables have a short life cycle. How to reasonably configure the proportio

After the RegionServer node is expanded, You need to migrate some of the original Region to the new RegionServer to enable Load Balancing for each RegionServer. To improve the HBase write performance after one node is added, You need to evenly distribute the RowKey in the List to all existing Region databases during each put operation, to achieve the concurrent processing of all regionservers. The following

Brief introduction Many Java applications today rely on a complex set of distributed dependencies and moving parts. Many external factors can have an impact on the performance and availability of your application. These effects are largely not completely eliminated or resolved, and are difficult to simulate accurately in a prebuilt environment. Stuff happens. However, you can create and maintain a comprehensive system to monitor the entire ecosystem

SNMP collects data on server performance, such as CPU usage, memory usage, CPU load, number of system processes, disk space usage, network traffic, disk I/O, and so on, while using the monitoring treasure to process these data can be visually displayed. The following describes how to configureInstalling SNMP1, tohttp://www.net-snmp.org/Download the latest version of the SNMP installation package.1. wgethttp