cloudfront access logs

Originally, maptail was able to deploy on a web server host to view web service access logs in real time. Now there are multiple web servers that need to be monitored, which is a little troublesome. Therefore, one machine is used to monitor multiple web servers at the same time, because the essence of maptail is a demonstration of tail-f, we can combine this, that is, maptail only needs to have access_log,

Oracle alarm logs are frequently seen. The following describes how to access oracle alarm logs by using oracle External tables. 1. Create a user and grant permissions SQL>createusercheckeridentifiedbypassworddefaulttablespaceusersquotaunlimitedonusers; SQL>grantconnect,resource,createanydirectorytochecker; 2. Create a directory object SQL>showp

In Linux, we can use crontab to regularly move access. log to the backup directory. At the same time, we can signal USR1 to the nginx main process to generate a new log file. Before writing the script, make the following assumptions: the log file is: usrlocalnginxlogsaccess. lognginx main process id is saved in the file: usrlocaln In Linux, we can useCrontab, Regularly move access. log to the backup directo

Python parsing Web Access logs Common Log Format127.0.0.1--[14/may/2017:12:45:29 +0800] "get/index.html http/1.1" 200 4286Remote-host IP Request time TimeZone Method Resource Protocol status code send bytes Combined log Format127.0.0.1--[14/may/2017:12:51:13 +0800] "get/index.html http/1.1", 4286 "HTTP://127.0.0.1/" "mozilla/5.0 (Windows N T 6.1; Win64; x64) applewebkit/537.36 (khtml, like Gecko) c

#!/bin/sh# feature: Find all access logs for a specific IP from the IP list and generate the corresponding files # #mkdir handle> com.ip.txtfunctionhandle () { #echo "egrep "$" 0602.log>handle/$n. txt " egrep "$" 0602.log>handle/$1.txt time= ' wc-lhandle/$n .txt|awk ' {print$1} ' mvhandle/$1.txthandle/$1__${time}.txt #sed -i "/$1/d" 0602.log echo$1>>com.ip.txtsleep 0.02}m=0fornin ' catdiff.ip ' dom= ' ex

1, the number of IP direct output display:Cat Access_log_2011_06_26.log |awk ' {print $} ' |uniq-c|wc-l2, the number of IP output to the text display:Cat Access_log_2011_06_26.log |awk ' {print $} ' |uniq-c|wc-l > Ip.txtSummary: If a single access log is larger than 2G, the system load will rise when viewed with this command, so do not view when the server is under high load, preferably in a low load time period. The above is one of the company's adve

Nginx does not log unwanted access logsNginx does not record the log pointing to the elements of the site, why do you need to do so, under what circumstances need to do this:When calculating the log PV, generally do not need to statistics picture element of the log, because, open a Web page is a PV, when the RS server does not want to recordA useless log from the front-end that balances health checks.CaseThe following is a configuration of resources t

(msg["from"],mail_list,msg.as_string ()) smtp.close ()if __name__ = = ' __ Main__ ': run () filename=' out.csv ' cmd = ' iconv-f utf8-t GB18030%s-o%s.bak mv%s.bak %s '% (filename,filename,filename,filename) os.system (cmd) content= ' Dear all: Subject = ' [Analysis] National creation data IP analysis statistics ' sendMail (html=content,emailaddress=' [email protected] ', Mailsubject=subject) V. Results of implementationUTF-8IP:41.42.97.104 Country: Egypt visits: 2ip:99.122.189.

Vim/usr/local/apache2/conf/extra/httpd-vhosts.confErrorlog "|/usr/local/apache/bin/rotatelogs-l/usr/local/apache/logs/oem.discuz.qq.com-error_%y%m%d.log 86400"Setenvif Request_uri ". *\.gif$" Image-requestSetenvif Request_uri ". *\.jpg$" Image-requestSetenvif Request_uri ". *\.png$" Image-requestSetenvif Request_uri ". *\.bmp$" Image-requestSetenvif Request_uri ". *\.swf$" Image-requestSetenvif Request_uri ". *\.js$" Image-requestSetenvif Request_uri

tools that come with the system are needed here. IIS Server installation automatically installs some scripts to configure IIS under C:\Inetpub\AdminScript, such as Adsutil.vbs, Chaccess.vbs, Mkwebdir.vbs, Disptree.vbs and so on, we can modify the permissions of the Web directory through these scripts.Under command line shell, enter "cscript.exe adsutil.vbs enum w3svc/1/Root "To see the configuration of the Web server. Choose a virtual directory test and give it writable permissions: "Chaccess-a

By default, the Tomcat application directory is the WebApps directory under the Tomcat base directory, the http://ip/project name is required to access the project after accessing the WebApps directory, and if you want to configure Server.xml directly, modify Create multiple virtual hosts from a domain name Dation= "false" >Ation= "false" > By default, Tomcat defines the access log to

deleted or replaced, but does not affect our analysis results, of course, the format of what is not important, nginx access logs can be customized, each company may be slightly different, so to understand the script content, And through their own changes applied to their own work is the focus, I give the log format is a reference, I bet you see on your company server log format is certainly not the same as

the decompressed folder docs, tools, and all the cgi-bin folders under wwwroot to the createdFolder, new WEB-INF directory under AWStats, copy cgi-bin to WEB-INF directory, under WEB-INF directoryThe content of the new web. xml file isXmlns: xsi = "http://www.w3.org/2001/XMLSchema-instance"Xsi: schemalocation = "http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd"Version = "2.4"> Tomcat6.0 executable is Perl by default, but environment variables are set.The full path

Filter the invalid ip addresses in the access log and the script for regularly updating the ip addresses of the robot: www.2cto.com #! /Bin/sh # update the company IP address on a regular basis to filter # author: FelixZhang # date: 2012-12-29filedir/opt/logdata/companyipadate $ (date-d quot; today quot ;... filter the invalid ip addresses in the access log and the script for regularly updating the ip add

Use Python to parse Nginx access logs, split them according to the Nginx log format, and store them in the MySQL database.One, Nginx access log format is as follows: Copy Code code as follows: $remote _addr-$remote _user [$time _local] "$request" $status $body _bytes_sent "$http _referer" "$http _user_agent" "$http _x_forwarded_for "' #使用的是nginx默认日

Objective After the WAF is on line, the most processed is the false positives elimination. There are a number of reasons for false positives, such as allowing the client to submit too many cookies when the Web application source code is written, such as the number of individual parameter submissions is too large. After reducing the false positives to an acceptable range, you should also focus on false negatives. WAF is not God, and any WAF can be bypassed. So you also need to locate the missed

/db/HTTP/1.1"This is the most useful one. First, it tells our server to receive a GET request, second, the resource path of the client request, and third, HTTP/1.1 when the client uses the protocol, the entire format is "% m % U % q % H", that is, "request method/access path/protocol"(6) 200This is a status code sent back to the client by the server. it tells us whether the client's request is successful, or is redirected, or what kind of error is enc

#/bin/bash#日志文件Logfile= '/var/log/nginx/access.log 'Hours=1#开始时间Start_time=date -d "-$hours hour" +"%H:%M:%S"#echo $start _time#结束时间Stop_time=date +"%H:%M:%S"#echo $stop _time#过滤出单位之间内的日志并统计最高ip数Array= ($ (TAC $logfile | awk-v st= "$start _time"-v et= "$stop _time" ' {t=substr ($4,rstart+14,21); if (t>=st t Lt;=et) {print $}} ' \| awk ' {print '} ' | Sort | uniq-c | Sort-nr |awk ' {if ($ >) {print $}} ' |awk ' {print $} ')WaitFor all in ${array[@]}DoEcho $eachfirewall-cmd --permanent --add-rich

more than 100 times in the log#cat Access_log | Cut-d '-F 7 | Sort |uniq-c | awk ' {if (>) print $} ' | LessStatistics a URL, the number of visits per day#cat access_log|grep ' 12/aug/2009 ' |grep '/images/index/e1.gif ' |wc|awk ' {print $} 'Most visited pages in the first five days#cat Access_log|awk ' {print $7} ' |uniq-c |sort-n-r|head-20See what the IP is doing in the log.#cat Access_log | grep 218.66.36.119| awk ' {print ' \ t ' $7} ' | Sort | uniq-c | Sort-nr | LessList files that have be

protected] py]$ hive--service hiveserverStarting Hive Thrift Server3) Write the query script on node29:#!/usr/bin/envpython#coding:utf-8# Find the CDN log for the specified time period, the top 10 URLs visited;importsysimportosimport stringimportreimportmysqldb# loading hive Python-related library files; sys.path.append ('/usr/local/hive_py ') fromhive _serviceimportthrifthivefromhive_service.ttypesimporthiveserverexceptionfrom thriftimportThriftfromthrift.transportimportTSocketfrom Thrift.tran