If the existing data:
{"@timestamp": "2018-10-13t21:55:58+08:00", "REMOTE_ADDR": "100.120.34.3", "Referer": "-", "Request": "get/api/gourd/ Activeupload?idfa=58237fa9-a1b3-4202-b5f3-9536983119e5&appid=1410137206&source=rehulu HTTP/1.1 "," status ": $," request_time ": 0.076," Cookie ":"-"," host ":" cms.369wan.com "," bytes ":" The Agent ":" mozilla/5.0 "(X11; Ubuntu; Linux x86_64; rv:18.0) gecko/20100101 firefox/18.0 "," proxy_x_forwarded ":" 139.129.97.187, 100.120.34.3 "" Upstr_addr ":" 127.0.0.1:9000 "," Upstr_host ":"-"," Ups_resp_time ": 0.076}
{"@timestamp": "2018-10-13t21:56:06+08:00", "REMOTE_ADDR": "100.120.34.101", "Referer": "-", "Request": "get/api/ Gourd/activeupload?idfa=e9d7f87a-9042-46b4-82e8-e5f64b74466b&appid=1410137206&source=rehulu HTTP/1.1 "," Status ": $," request_time ": 0.076," Cookie ":"-"," host ":" cms.369wan.com "," bytes ":", "" Agent ": mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) gecko/20100101 firefox/18.0 "," proxy_x_forwarded ":" 139.129.97.187, 100.120.34.101 "" Upstr_addr ":" 127.0.0.1:9000 "," Upstr_host ":"-"," Ups_resp_time ": 0.076}
{"@timestamp": "2018-10-13t21:56:08+08:00", "REMOTE_ADDR": "100.120.34.29", "Referer": "-", "Request": "Get/api/gourd /activeupload?idfa=d5b924f3-7d25-4b52-bae9-3270b08ea32d&appid=1410137206&source=rehulu HTTP/1.1 "," Status ": $," request_time ": 0.075," Cookie ":"-"," host ":" cms.369wan.com "," bytes ":", "" Agent ": mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) gecko/20100101 firefox/18.0 "," proxy_x_forwarded ":" 139.129.97.187, 100.120.34.29 "" Upstr_addr ":" 127.0.0.1:9000 "," Upstr_host ":"-"," Ups_resp_time ": 0.075}
{"@timestamp": "2018-10-13t21:56:10+08:00", "REMOTE_ADDR": "100.120.34.75", "Referer": "-", "Request": "Get/api/gourd /activeupload?idfa=d166459d-e823-4847-9094-6f4bf90625b2&appid=1410137206&source=rehulu HTTP/1.1 "," Status ": $," request_time ": 0.078," Cookie ":"-"," host ":" cms.369wan.com "," bytes ":", "" Agent ": mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) gecko/20100101 firefox/18.0 "," proxy_x_forwarded ":" 139.129.97.187, 100.120.34.75 "" Upstr_addr ":" 127.0.0.1:9000 "," Upstr_host ":"-"," Ups_resp_time ": 0.078}
{"@timestamp": "2018-10-13t21:56:18+08:00", "REMOTE_ADDR": "100.120.34.39", "Referer": "-", "Request": "Get/api/gourd /activeupload?idfa=08c65c3b-eed2-4a65-b0c1-67fc7fb78e18&appid=1410137206&source=rehulu HTTP/1.1 "," Status ": $," request_time ": 0.082," Cookie ":"-"," host ":" cms.369wan.com "," bytes ":", "" Agent ": mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) gecko/20100101 firefox/18.0 "," proxy_x_forwarded ":" 139.129.97.187, 100.120.34.39 "" Upstr_addr ":" 127.0.0.1:9000 "," Upstr_host ":"-"," Ups_resp_time ": 0.082}
{"@timestamp": "2018-10-13t21:56:31+08:00", "REMOTE_ADDR": "100.120.34.68", "Referer": "-", "Request": "Get/api/gourd /activeupload?idfa=d5b924f3-7d25-4b52-bae9-3270b08ea32d&appid=1410137206&source=rehulu HTTP/1.1 "," Status ": $," request_time ": 0.079," Cookie ":"-"," host ":" cms.369wan.com "," bytes ":", "" Agent ": mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) gecko/20100101 firefox/18.0 "," proxy_x_forwarded ":" 139.129.97.187, 100.120.34.68 "" Upstr_addr ":" 127.0.0.1:9000 "," Upstr_host ":"-"," Ups_resp_time ": 0.079}
Temporarily Save as Tmp.log
Awk-f ' idfa= ' {print $} ' Tmp.log
The following results appear:
58237fa9-a1b3-4202-b5f3-9536983119e5&appid=1410137206&source=rehulu HTTP/1.1 "," status ": $," request_ Time ": 0.076," Cookie ":"-"," host ":" cms.369wan.com "," bytes ":," Agent ":" mozilla/5.0 "(X11; Ubuntu; Linux x86_64; rv:18.0) gecko/20100101 firefox/18.0 "," proxy_x_forwarded ":" 139.129.97.187, 100.120.34.3 "" Upstr_addr ":" 127.0.0.1:9000 "," Upstr_host ":"-"," Ups_resp_time ": 0.076}
E9d7f87a-9042-46b4-82e8-e5f64b74466b&appid=1410137206&source=rehulu HTTP/1.1 "," status ": $," request_ Time ": 0.076," Cookie ":"-"," host ":" cms.369wan.com "," bytes ":," Agent ":" mozilla/5.0 "(X11; Ubuntu; Linux x86_64; rv:18.0) gecko/20100101 firefox/18.0 "," proxy_x_forwarded ":" 139.129.97.187, 100.120.34.101 "" Upstr_addr ":" 127.0.0.1:9000 "," Upstr_host ":"-"," Ups_resp_time ": 0.076}
D5b924f3-7d25-4b52-bae9-3270b08ea32d&appid=1410137206&source=rehulu HTTP/1.1 "," status ": $," request_ Time ": 0.075," Cookie ":"-"," host ":" cms.369wan.com "," bytes ":," Agent ":" mozilla/5.0 "(X11; Ubuntu; Linux x86_64; rv:18.0) gecko/20100101 firefox/18.0 "," proxy_x_forwarded ":" 139.129.97.187, 100.120.34.29 "" Upstr_addr ":" 127.0.0.1:9000 "," Upstr_host ":"-"," Ups_resp_time ": 0.075}
Perform
Awk-f ' idfa= ' {print $} ' Tmp.log | Awk-f ' &source= ' {print $} '
The following results appear:
58237fa9-a1b3-4202-b5f3-9536983119e5&appid=1410137206
e9d7f87a-9042-46b4-82e8-e5f64b74466b&appid=1410137206
d5b924f3-7d25-4b52-bae9-3270b08ea32d&appid=1410137206
Perform
Awk-f ' idfa= ' {print $} ' Tmp.log | Awk-f ' &source= ' {print $} ' | Sort (This step puts the results together in the same way)
The following results appear:
58237fa9-a1b3-4202-b5f3-9536983119e5&appid=1410137206
e9d7f87a-9042-46b4-82e8-e5f64b74466b&appid=1410137206
d5b924f3-7d25-4b52-bae9-3270b08ea32d&appid=1410137206
Perform
Awk-f ' idfa= ' {print $} ' cms_\ (2\). Log | Awk-f ' &source= ' {print $} ' |sort| Uniq-c
The final result (number of times and each string) appears:
1 58237fa9-a1b3-4202-b5f3-9536983119e5&appid=1410137206
1 e9d7f87a-9042-46b4-82e8-e5f64b74466b&appid=1410137206
1 d5b924f3-7d25-4b52-bae9-3270b08ea32d&appid=1410137206
Number of occurrences of some specific string grouping statistics under CentOS for a file