How to view QPS on Linux

How to view QPS on Linux

Background: there is a recommendation service online with a large log volume. Check if the qps is too high.

Question: How can I view the qps of a service based on logs?

Tail-f XXX. log:
[8708-10 14:51:44 638 INFO] [async task worker [61] recommend. components. KeywordService [87]-cateid = 252 pageNum = 1
[8708-10 14:51:44 666 INFO] [async task worker [62] recommend. components. KeywordService [87]-cateid = 42205
[8708-10 14:51:44 673 INFO] [async task worker [0] recommend. components. KeywordService [87]-cateid = 29 pageNum = 2
[8708-10 14:51:44 677 INFO] [async task worker [1] recommend. components. KeywordService [87]-cateid = 252 pageNum = 3

In the log specification, there is a column "request time". You can use this "request time" to estimate the service qps. The steps are as follows:
(1) first, find a log that makes a request have only one line. The common tool is grep. In this example, grep recommend. components. KeywordService is used to obtain the result. A request corresponds to a line of log.
(2) extract the Time column. The common tool is cut or awk. Here we will introduce cut (you can go to linux to man)
-D parameter, separated by a certain character
-F parameter, which is used to retrieve the column number after separation
In this example, the time is in the second column after being separated by spaces.
The result is as follows:
14:51:44
14:51:44
14:51:45
14:51:45
14:51:46
14:51:46
(3) deduplicate and count the result. The commonly used tool is uniq and the parameter is-c.

Therefore, the entire shell command is:
Command: tail-f XXX. log | grep recommend. components. KeywordService | cut-d ''-f2 | cut-d': '-f3 | uniq-c
Description: Incremental fetch | one request is used to fetch a row | extract time | extract seconds | deduplicate count
The result is:
136 43
126 44
115 45
131 46
132 47
We can see that there are 136 logs in total at 14:51:43.
44, with 126 logs
45. There are 115 logs
...
Conclusion: The qps of a single machine in this module is around -.

This article permanently updates the link address: